Package org.apache.sysds.runtime.controlprogram.paramserv.dp

Class DataPartitionerSparkAggregator

java.lang.Object

org.apache.sysds.runtime.controlprogram.paramserv.dp.DataPartitionerSparkAggregator

All Implemented Interfaces:

Serializable, org.apache.spark.api.java.function.PairFunction<scala.Tuple2<Integer,LinkedList<scala.Tuple2<Long,scala.Tuple2<MatrixBlock,MatrixBlock>>>>,Integer,scala.Tuple2<MatrixBlock,MatrixBlock>>

public class DataPartitionerSparkAggregator
extends Object
implements org.apache.spark.api.java.function.PairFunction<scala.Tuple2<Integer,LinkedList<scala.Tuple2<Long,scala.Tuple2<MatrixBlock,MatrixBlock>>>>,Integer,scala.Tuple2<MatrixBlock,MatrixBlock>>

See Also:

• Serialized Form

Constructor Summary

Constructors

Constructor	Description
DataPartitionerSparkAggregator()
DataPartitionerSparkAggregator(long fcol, long lcol)

Method Summary

Modifier and Type	Method	Description
scala.Tuple2<Integer,scala.Tuple2<MatrixBlock,MatrixBlock>>	call(scala.Tuple2<Integer,LinkedList<scala.Tuple2<Long,scala.Tuple2<MatrixBlock,MatrixBlock>>>> input)	Row-wise combine the matrix

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

DataPartitionerSparkAggregator

public DataPartitionerSparkAggregator()

DataPartitionerSparkAggregator

public DataPartitionerSparkAggregator(long fcol,
 long lcol)

Method Details

call

public scala.Tuple2<Integer,scala.Tuple2<MatrixBlock,MatrixBlock>> call(scala.Tuple2<Integer,LinkedList<scala.Tuple2<Long,scala.Tuple2<MatrixBlock,MatrixBlock>>>> input)
                                                                 throws Exception

Row-wise combine the matrix

Specified by:

call in interface org.apache.spark.api.java.function.PairFunction<scala.Tuple2<Integer,LinkedList<scala.Tuple2<Long,scala.Tuple2<MatrixBlock,MatrixBlock>>>>,Integer,scala.Tuple2<MatrixBlock,MatrixBlock>>

Parameters:

input - workerID => ordered list [(rowBlockID, (features, labels))]

Returns:

workerID => [(features, labels)]

Throws:

Exception - Some exception