Package org.apache.sysds.runtime.controlprogram

Class ParForProgramBlock

java.lang.Object
org.apache.sysds.runtime.controlprogram.ProgramBlock
org.apache.sysds.runtime.controlprogram.ForProgramBlock
org.apache.sysds.runtime.controlprogram.ParForProgramBlock
All Implemented Interfaces:
ParseInfo
public class ParForProgramBlock extends ForProgramBlock
The ParForProgramBlock has the same execution semantics as a ForProgamBlock but executes the independent iterations in parallel. See ParForStatementBlock for the loop dependency analysis. At runtime level, iterations are guaranteed to be completely independent. NEW FUNCTIONALITIES TODO: reduction variables (operations: +=, -=, /=, *=, min, max) TODO: papply(A,1:2,FUN) language construct (compiled to ParFOR) via DML function repository => modules OK, but second-order functions required

  • Field Details

    • OPTIMIZE

      public static final boolean OPTIMIZE
      See Also:

    • USE_PB_CACHE

      public static final boolean USE_PB_CACHE
      See Also:

    • USE_RANGE_TASKS_IF_USEFUL

      public static final boolean USE_RANGE_TASKS_IF_USEFUL
      See Also:

    • USE_STREAMING_TASK_CREATION

      public static final boolean USE_STREAMING_TASK_CREATION
      See Also:

    • ALLOW_NESTED_PARALLELISM

      public static final boolean ALLOW_NESTED_PARALLELISM
      See Also:

    • CONVERT_NESTED_REMOTE_PARFOR

      public static final boolean CONVERT_NESTED_REMOTE_PARFOR
      See Also:

    • USE_PARALLEL_RESULT_MERGE

      public static final boolean USE_PARALLEL_RESULT_MERGE
      See Also:

    • USE_PARALLEL_RESULT_MERGE_REMOTE

      public static final boolean USE_PARALLEL_RESULT_MERGE_REMOTE
      See Also:

    • CREATE_UNSCOPED_RESULTVARS

      public static final boolean CREATE_UNSCOPED_RESULTVARS
      See Also:

    • ALLOW_REUSE_PARTITION_VARS

      public static boolean ALLOW_REUSE_PARTITION_VARS

    • WRITE_REPLICATION_FACTOR

      public static final int WRITE_REPLICATION_FACTOR
      See Also:

    • MAX_RETRYS_ON_ERROR

      public static int MAX_RETRYS_ON_ERROR

    • FORCE_CP_ON_REMOTE_SPARK

      public static final boolean FORCE_CP_ON_REMOTE_SPARK
      See Also:

    • LIVEVAR_AWARE_EXPORT

      public static final boolean LIVEVAR_AWARE_EXPORT
      See Also:

    • RESET_RECOMPILATION_FLAGs

      public static final boolean RESET_RECOMPILATION_FLAGs
      See Also:

    • ALLOW_BROADCAST_INPUTS

      public static boolean ALLOW_BROADCAST_INPUTS

    • COPY_EVAL_FUNCTIONS

      public static final boolean COPY_EVAL_FUNCTIONS
      See Also:

    • PARFOR_FNAME_PREFIX

      public static final String PARFOR_FNAME_PREFIX
      See Also:

    • PARFOR_MR_TASKS_TMP_FNAME

      public static final String PARFOR_MR_TASKS_TMP_FNAME
      See Also:

    • PARFOR_MR_RESULT_TMP_FNAME

      public static final String PARFOR_MR_RESULT_TMP_FNAME
      See Also:

    • PARFOR_MR_RESULTMERGE_FNAME

      public static final String PARFOR_MR_RESULTMERGE_FNAME
      See Also:

    • PARFOR_DATAPARTITIONS_FNAME

      public static final String PARFOR_DATAPARTITIONS_FNAME
      See Also:

    • PARFOR_COUNTER_GROUP_NAME

      public static final String PARFOR_COUNTER_GROUP_NAME
      See Also:

  • Constructor Details

    • ParForProgramBlock

      public ParForProgramBlock(Program prog, String iterPredVar, HashMap<String,String> params, ArrayList<ParForStatementBlock.ResultVar> resultVars)

    • ParForProgramBlock

      public ParForProgramBlock(int ID, Program prog, String iterPredVar, HashMap<String,String> params, ArrayList<ParForStatementBlock.ResultVar> resultVars)
      ParForProgramBlock constructor. It reads the specified parameter settings, where defaults for non-specified parameters have been set in ParForStatementBlock.validate(). Furthermore, it generates the IDs for the ParWorkers.
      Parameters:
      ID - parfor program block id
      prog - runtime program
      iterPredVar - ?
      params - map of parameters
      resultVars - list of result variable names

  • Method Details

    • resetWorkerIDs

      public static void resetWorkerIDs()

    • getID

      public long getID()

    • getExecMode

      public ParForProgramBlock.PExecMode getExecMode()

    • getParForParams

      public HashMap<String,String> getParForParams()

    • getParForParam

      public String getParForParam(String key)

    • getResultVariables

      public ArrayList<ParForStatementBlock.ResultVar> getResultVariables()

    • disableOptimization

      public void disableOptimization()

    • getOptimizationMode

      public ParForProgramBlock.POptMode getOptimizationMode()

    • setOptimizationMode

      public void setOptimizationMode(ParForProgramBlock.POptMode mode)

    • getDegreeOfParallelism

      public int getDegreeOfParallelism()

    • setDegreeOfParallelism

      public void setDegreeOfParallelism(int k)

    • isDegreeOfParallelismFixed

      public boolean isDegreeOfParallelismFixed()

    • setDegreeOfParallelismFixed

      public void setDegreeOfParallelismFixed(boolean flag)

    • setCPCaching

      public void setCPCaching(boolean flag)

    • setRuntimePiggybacking

      public void setRuntimePiggybacking(boolean flag)

    • setExecMode

      public void setExecMode(ParForProgramBlock.PExecMode mode)

    • setTaskPartitioner

      public void setTaskPartitioner(ParForProgramBlock.PTaskPartitioner partitioner)

    • setTaskSize

      public void setTaskSize(long tasksize)

    • setDataPartitioner

      public void setDataPartitioner(ParForProgramBlock.PDataPartitioner partitioner)

    • enableColocatedPartitionedMatrix

      public void enableColocatedPartitionedMatrix(String varname)

    • setTransposeSparseColumnVector

      public void setTransposeSparseColumnVector(boolean flag)

    • setPartitionReplicationFactor

      public void setPartitionReplicationFactor(int rep)

    • setExportReplicationFactor

      public void setExportReplicationFactor(int rep)

    • disableJVMReuse

      public void disableJVMReuse()

    • setResultMerge

      public void setResultMerge(ParForProgramBlock.PResultMerge merge)

    • setRecompileMemoryBudget

      public void setRecompileMemoryBudget(double localMem)

    • setSparkRepartitionVariables

      public void setSparkRepartitionVariables(Collection<String> vars)

    • getSparkRepartitionVariables

      public Collection<String> getSparkRepartitionVariables()

    • setSparkEagerCacheVariables

      public void setSparkEagerCacheVariables(Collection<String> vars)

    • getNumIterations

      public long getNumIterations()

    • hasFunctions

      public boolean hasFunctions()

    • execute

      public void execute(ExecutionContext ec)
      Description copied from class: ProgramBlock
      Executes this program block (incl recompilation if required).
      Overrides:
      execute in class ForProgramBlock
      Parameters:
      ec - execution context

    • createResultMerge

      public static ResultMerge<?> createResultMerge(ParForProgramBlock.PResultMerge prm, CacheableData<?> out, CacheableData<?>[] in, String fname, boolean accum, int numThreads, ExecutionContext ec)

    • printBlockErrorLocation

      public String printBlockErrorLocation()
      Overrides:
      printBlockErrorLocation in class ForProgramBlock