org.apache.sysds.runtime.controlprogram.parfor

## Class RemoteParForSpark

- java.lang.Object
- - org.apache.sysds.runtime.controlprogram.parfor.RemoteParForSpark

-     public class RemoteParForSpark
      extends Object

  This class serves two purposes: (1) isolating Spark imports to enable running in environments where no Spark libraries are available, and (2) to follow the same structure as the parfor remote_mr job submission. NOTE: currently, we still exchange inputs and outputs via hdfs (this covers the general case if data already resides in HDFS, in-memory data, and partitioned inputs; also, it allows for pre-aggregation by overwriting partial task results with pre-paggregated results from subsequent iterations) TODO reducebykey on variable names

- - ### Constructor Summary

    Constructors
    Constructor and Description
    `RemoteParForSpark()`

  - ### Method Summary

    All Methods Static Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `static RemoteParForJobReturn` | `runJob(long pfid, String prog, HashMap<String,byte[]> clsMap, List<Task> tasks, ExecutionContext ec, Set<String> brVars, List<ParForStatementBlock.ResultVar> resultVars, boolean cpCaching, int numMappers, boolean topLevelPF)`

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Constructor Detail

    - #### RemoteParForSpark

          public RemoteParForSpark()

  - ### Method Detail

    - #### runJob

          public static RemoteParForJobReturn runJob(long pfid,
                                                     String prog,
                                                     HashMap<String,byte[]> clsMap,
                                                     List<Task> tasks,
                                                     ExecutionContext ec,
                                                     Set<String> brVars,
                                                     List<ParForStatementBlock.ResultVar> resultVars,
                                                     boolean cpCaching,
                                                     int numMappers,
                                                     boolean topLevelPF)

Copyright © 2021 The Apache Software Foundation. All rights reserved.