Uses of Class
org.apache.sysds.resource.cost.IOCostUtils.IOMetrics
Packages that use IOCostUtils.IOMetrics
-
Uses of IOCostUtils.IOMetrics in org.apache.sysds.resource.cost
Methods in org.apache.sysds.resource.cost with parameters of type IOCostUtils.IOMetricsModifier and TypeMethodDescriptionstatic doubleSparkCostUtils.getAggUnaryInstTime(UnarySPInstruction inst, VarStats input, VarStats output, IOCostUtils.IOMetrics executorMetrics) static doubleSparkCostUtils.getAppendInstTime(AppendSPInstruction inst, VarStats input1, VarStats input2, VarStats output, IOCostUtils.IOMetrics driverMetrics, IOCostUtils.IOMetrics executorMetrics) static doubleCPCostUtils.getBinaryInstTime(BinaryCPInstruction inst, VarStats input1, VarStats input2, VarStats weights, VarStats output, IOCostUtils.IOMetrics metrics) static doubleSparkCostUtils.getBinaryInstTime(SPInstruction inst, VarStats input1, VarStats input2, VarStats output, IOCostUtils.IOMetrics driverMetrics, IOCostUtils.IOMetrics executorMetrics) static doubleCPCostUtils.getBuiltinNaryInstTime(BuiltinNaryCPInstruction inst, VarStats[] inputs, VarStats output, IOCostUtils.IOMetrics metrics) static doubleSparkCostUtils.getCastInstTime(CastSPInstruction inst, VarStats input, VarStats output, IOCostUtils.IOMetrics executorMetrics) static doubleSparkCostUtils.getCentralMomentInstTime(CentralMomentSPInstruction inst, VarStats input, VarStats weights, VarStats output, IOCostUtils.IOMetrics executorMetrics) static doubleCPCostUtils.getComputationInstTime(ComputationCPInstruction inst, VarStats input1, VarStats input2, VarStats input3, VarStats input4, VarStats output, IOCostUtils.IOMetrics metrics) static doubleCPCostUtils.getCPUTime(long nflop, IOCostUtils.IOMetrics driverMetrics, VarStats output, VarStats... inputs) static doubleSparkCostUtils.getCPUTime(long nflop, int numPartitions, IOCostUtils.IOMetrics executorMetrics, RDDStats output, RDDStats... inputs) Computes an estimate for the time needed by the CPU to execute (including memory access) an instruction by providing number of floating operations.static doubleSparkCostUtils.getCtableInstTime(CtableSPInstruction tableInst, VarStats input1, VarStats input2, VarStats input3, VarStats output, IOCostUtils.IOMetrics executorMetrics) static doubleCPCostUtils.getDataGenCPInstTime(UnaryCPInstruction inst, VarStats output, IOCostUtils.IOMetrics metrics) static doubleIOCostUtils.getFileSystemReadTime(VarStats stats, IOCostUtils.IOMetrics metrics) Estimates the read time for a file on HDFS or S3 by the Control Programstatic doubleIOCostUtils.getFileSystemWriteTime(VarStats stats, IOCostUtils.IOMetrics metrics) Estimates the time for writing a file to HDFS or S3.static doubleIOCostUtils.getHadoopReadTime(VarStats stats, IOCostUtils.IOMetrics metrics) Estimates the read time for a file on HDFS or S3 by Spark cluster.static doubleIOCostUtils.getHadoopWriteTime(VarStats stats, IOCostUtils.IOMetrics metrics) Estimates the write time for a file on HDFS or S3 by Spark cluster.static doubleSparkCostUtils.getIndexingInstTime(IndexingSPInstruction inst, VarStats input1, VarStats input2, VarStats output, IOCostUtils.IOMetrics driverMetrics, IOCostUtils.IOMetrics executorMetrics) static doubleSparkCostUtils.getMatMulChainInstTime(MapmmChainSPInstruction inst, VarStats input1, VarStats input2, VarStats input3, VarStats output, IOCostUtils.IOMetrics driverMetrics, IOCostUtils.IOMetrics executorMetrics) static doubleSparkCostUtils.getMatMulInstTime(BinarySPInstruction inst, VarStats input1, VarStats input2, VarStats output, IOCostUtils.IOMetrics driverMetrics, IOCostUtils.IOMetrics executorMetrics) static doubleIOCostUtils.getMemReadTime(RDDStats stats, IOCostUtils.IOMetrics metrics) Estimate time to scan distributed data sets in memory on Spark.static doubleIOCostUtils.getMemReadTime(VarStats stats, IOCostUtils.IOMetrics metrics) Estimate time to scan object in memory in CP.static doubleIOCostUtils.getMemWriteTime(RDDStats stats, IOCostUtils.IOMetrics metrics) Estimate time to write distributed data set on memory in CP.static doubleIOCostUtils.getMemWriteTime(VarStats stats, IOCostUtils.IOMetrics metrics) Estimate time to write object to memory in CP.static doubleCPCostUtils.getMultiReturnBuiltinInstTime(MultiReturnBuiltinCPInstruction inst, VarStats input, VarStats[] outputs, IOCostUtils.IOMetrics metrics) static doubleCPCostUtils.getParameterizedBuiltinInstTime(ParameterizedBuiltinCPInstruction inst, VarStats input, VarStats output, IOCostUtils.IOMetrics metrics) static doubleSparkCostUtils.getParameterizedBuiltinInstTime(ParameterizedBuiltinSPInstruction paramInst, VarStats input1, VarStats input2, VarStats output, IOCostUtils.IOMetrics driverMetrics, IOCostUtils.IOMetrics executorMetrics) static doubleSparkCostUtils.getQSortInstTime(QuantileSortSPInstruction inst, VarStats input, VarStats weights, VarStats output, IOCostUtils.IOMetrics executorMetrics) static doubleSparkCostUtils.getQuaternaryInstTime(QuaternarySPInstruction quatInst, VarStats input1, VarStats input2, VarStats input3, VarStats output, IOCostUtils.IOMetrics driverMetrics, IOCostUtils.IOMetrics executorMetrics) static doubleSparkCostUtils.getRandInstTime(String opcode, int randType, VarStats output, IOCostUtils.IOMetrics executorMetrics) static doubleSparkCostUtils.getReblockInstTime(String opcode, VarStats input, VarStats output, IOCostUtils.IOMetrics executorMetrics) static doubleSparkCostUtils.getReorgInstTime(UnarySPInstruction inst, VarStats input, VarStats output, IOCostUtils.IOMetrics executorMetrics) static doubleIOCostUtils.getSparkCollectTime(RDDStats output, IOCostUtils.IOMetrics driverMetrics, IOCostUtils.IOMetrics executorMetrics) Estimates the time for collecting Spark Job output; The output RDD is transferred to the Spark driver at the end of each ResultStage; time = transfer time (overlaps and dominates the read and deserialization times);static doubleIOCostUtils.getSparkParallelizeTime(RDDStats output, IOCostUtils.IOMetrics driverMetrics, IOCostUtils.IOMetrics executorMetrics) Estimates the time to parallelize a local object to Spark.static doubleIOCostUtils.getSparkShuffleReadStaticTime(RDDStats input, IOCostUtils.IOMetrics metrics) Estimates the time for reading distributed RDD input at the beginning of a Stage when a wide-transformation is partition preserving: only local disk readsstatic doubleIOCostUtils.getSparkShuffleReadTime(RDDStats input, IOCostUtils.IOMetrics metrics) Estimates the time for reading distributed RDD input at the beginning of a Stage; time = transfer time (overlaps and dominates the read and deserialization times); For simplification it is assumed that the whole dataset is shuffled;static doubleIOCostUtils.getSparkShuffleTime(RDDStats output, IOCostUtils.IOMetrics metrics, boolean withDistribution) Combines the shuffle write and read time since these are being typically added in one place to the general data transmission for instruction estimation.static doubleIOCostUtils.getSparkShuffleWriteTime(RDDStats output, IOCostUtils.IOMetrics metrics) Estimates the time for writing the RDD output to the local system at the end of a ShuffleMapStage; time = disk write time (overlaps and dominates the serialization time) The whole data set is being written to shuffle files even if 1 executor is utilized;static doubleSparkCostUtils.getTernaryInstTime(TernarySPInstruction tInst, VarStats input1, VarStats input2, VarStats input3, VarStats output, IOCostUtils.IOMetrics executorMetrics) static doubleSparkCostUtils.getTSMMInstTime(UnarySPInstruction inst, VarStats input, VarStats output, IOCostUtils.IOMetrics driverMetrics, IOCostUtils.IOMetrics executorMetrics) static doubleCPCostUtils.getUnaryInstTime(UnaryCPInstruction inst, VarStats input, VarStats weights, VarStats output, IOCostUtils.IOMetrics metrics) static doubleSparkCostUtils.getUnaryInstTime(String opcode, VarStats input, VarStats output, IOCostUtils.IOMetrics executorMetrics) static doubleCPCostUtils.getVariableInstTime(VariableCPInstruction inst, VarStats input, VarStats output, IOCostUtils.IOMetrics metrics)