Package org.apache.sysds.lops

## Class Lop

- java.lang.Object
- - org.apache.sysds.lops.Lop

- Direct Known Subclasses:
    `Append`, `AppendG`, `AppendGAlignedSP`, `AppendM`, `AppendR`, `Binary`, `BinaryM`, `BinaryScalar`, `BinaryUAggChain`, `CentralMoment`, `Checkpoint`, `Compression`, `CoVariance`, `CSVReBlock`, `Ctable`, `CumulativeOffsetBinary`, `CumulativePartialAggregate`, `Data`, `DataGen`, `DeCompression`, `DnnTransform`, `Federated`, `FunctionCallCP`, `GroupedAggregate`, `GroupedAggregateM`, `LeftIndex`, `Local`, `MapMult`, `MapMultChain`, `MatMultCP`, `MMCJ`, `MMRJ`, `MMTSJ`, `MMZip`, `Nary`, `ParameterizedBuiltin`, `PartialAggregate`, `PickByCount`, `PMapMult`, `PMMJ`, `ReBlock`, `RightIndex`, `SortKeys`, `SpoofFused`, `Sql`, `Ternary`, `TernaryAggregate`, `Transform`, `UAggOuterChain`, `Unary`, `UnaryCP`, `WeightedCrossEntropy`, `WeightedCrossEntropyR`, `WeightedDivMM`, `WeightedDivMMR`, `WeightedSigmoid`, `WeightedSigmoidR`, `WeightedSquaredLoss`, `WeightedSquaredLossR`, `WeightedUnaryMM`, `WeightedUnaryMMR`

      public abstract class Lop
      extends Object

  Base class for all Lops.

- - ### Nested Class Summary

    Nested Classes
    Modifier and Type | Class | Description
    --- | --- | ---
    `static class` | `Lop.SimpleInstType` | Lop types
    `static class` | `Lop.Type` |
    `static class` | `Lop.VisitStatus` |

  - ### Field Summary

    Fields
    Modifier and Type | Field | Description
    --- | --- | ---
    `int` | `_beginColumn` |
    `int` | `_beginLine` |
    `int` | `_endColumn` |
    `int` | `_endLine` |
    `String` | `_filename` |
    `static String` | `CP_CHILD_THREAD` |
    `static String` | `CP_ROOT_THREAD_ID` |
    `static String` | `DATATYPE_PREFIX` |
    `static String` | `FILE_SEPARATOR` |
    `static String` | `FRAME_VAR_NAME_PREFIX` |
    `static String` | `INSTRUCTION_DELIMITOR` |
    `static String` | `LITERAL_PREFIX` |
    `static String` | `MATRIX_VAR_NAME_PREFIX` |
    `static String` | `NAME_VALUE_SEPARATOR` |
    `static String` | `OPERAND_DELIMITOR` |
    `static String` | `PROCESS_PREFIX` |
    `static double` | `SAMPLE_FRACTION` |
    `static String` | `SCALAR_VAR_NAME_PREFIX` |
    `static String` | `UPDATE_INPLACE_PREFIX` |
    `static String` | `VALUETYPE_PREFIX` |
    `static String` | `VARIABLE_NAME_PLACEHOLDER` |

  - ### Constructor Summary

    Constructors
    Constructor | Description
    --- | ---
    `Lop​(Lop.Type t, Types.DataType dt, Types.ValueType vt)` | Constructor to be invoked by base class.

  - ### Method Summary

    All Methods Instance Methods Abstract Methods Concrete Methods
    Modifier and Type | Method | Description
    --- | --- | ---
    `void` | `activatePrefetch()` |
    `void` | `addInput​(Lop op)` | Method to add input to Lop
    `void` | `addOutput​(Lop op)` | Method to add output to Lop
    `void` | `addToDag​(Dag<Lop> dag)` | Method to recursively add LOPS to a DAG
    `boolean[]` | `createReachable​(int size)` |
    `AggBinaryOp.SparkAggType` | `getAggType()` | Method to get aggregate type if applicable.
    `int` | `getBeginColumn()` |
    `int` | `getBeginLine()` |
    `Lop` | `getBroadcastInput()` | Method to get the input to be broadcast.
    `Types.DataType` | `getDataType()` | get data type of the output that is produced by this lop
    `int` | `getEndColumn()` |
    `int` | `getEndLine()` |
    `Types.ExecType` | `getExecType()` | Method to get the execution type (CP, CP_FILE, MR, SPARK, GPU, FED, INVALID) of LOP
    `FEDInstruction.FederatedOutput` | `getFederatedOutput()` |
    `String` | `getFilename()` |
    `long` | `getID()` | Method to return the ID of LOP
    `ArrayList<Lop>` | `getInputs()` | Method to get input of Lops
    `String` | `getInstructions()` | Method should be overridden if needed
    `String` | `getInstructions​(int input, int rowl, int rowu, int coll, int colu, int leftRowDim, int leftColDim, int output)` |
    `String` | `getInstructions​(String output)` | Method should be overridden if needed
    `String` | `getInstructions​(String[] inputs, String outputs)` |
    `String` | `getInstructions​(String[] inputs, String[] outputs)` | Method should be overridden if needed
    `String` | `getInstructions​(String input1, String output)` | Method should be overridden if needed
    `String` | `getInstructions​(String input1, String input2, String output)` | Method should be overridden if needed
    `String` | `getInstructions​(String input1, String input2, String input3, String output)` | Method should be overridden if needed
    `String` | `getInstructions​(String input1, String input2, String input3, String input4, String output)` | Method should be overridden if needed
    `String` | `getInstructions​(String input1, String input2, String input3, String input4, String input5, String output)` | Method should be overridden if needed
    `String` | `getInstructions​(String input1, String input2, String input3, String input4, String input5, String input6, String output)` | Method should be overridden if needed
    `String` | `getInstructions​(String input1, String input2, String input3, String input4, String input5, String input6, String input7, String output)` |
    `int` | `getLevel()` |
    `OutputParameters` | `getOutputParameters()` | Method to get output parameters
    `ArrayList<Lop>` | `getOutputs()` | Method to get output of Lops
    `PrivacyConstraint` | `getPrivacyConstraint()` |
    `boolean` | `getProducesIntermediateOutput()` |
    `boolean[]` | `getReachable()` |
    `Lop.SimpleInstType` | `getSimpleInstructionType()` | Method should be overridden if needed
    `Lop.Type` | `getType()` | Method to get Lop type.
    `Types.ValueType` | `getValueType()` | get value type of the output that is produced by this lop
    `Lop.VisitStatus` | `getVisited()` | get visit status of node
    `boolean` | `isAllOutputsCP()` | Function that determines if all the outputs of a LOP are of CP execution types
    `boolean` | `isDataExecLocation()` |
    `boolean` | `isVariable()` | Function that determines if the output of a LOP is defined by a variable or not.
    `boolean` | `prefetchActivated()` |
    `String` | `prepInputOperand​(int index)` |
    `String` | `prepInputOperand​(String label)` |
    `String` | `prepOperand​(String label, Types.DataType dt, Types.ValueType vt)` | Method to prepare instruction operand with given parameters.
    `String` | `prepOperand​(String label, Types.DataType dt, Types.ValueType vt, boolean literal)` | Method to prepare instruction operand with given parameters.
    `String` | `prepOutputOperand()` |
    `String` | `prepOutputOperand​(int index)` |
    `String` | `prepOutputOperand​(String label)` |
    `String` | `prepScalarInputOperand​(String label)` |
    `String` | `prepScalarInputOperand​(Types.ExecType et)` |
    `String` | `prepScalarLabel()` | Function to prepare label for scalar inputs while generating instructions.
    `String` | `prepScalarOperand​(Types.ExecType et, String label)` | Function to be used in creating instructions for creating scalar operands.
    `String` | `printErrorLocation()` |
    `int` | `removeConsumer()` |
    `void` | `removeOutput​(Lop op)` | Method to remove output from Lop
    `void` | `replaceInput​(Lop oldInp, Lop newInp)` | Method to replace an input to a Lop
    `void` | `resetVisitStatus()` |
    `void` | `setAllPositions​(String filename, int blp, int bcp, int elp, int ecp)` |
    `void` | `setBeginColumn​(int passed)` |
    `void` | `setBeginLine​(int passed)` |
    `void` | `setConsumerCount​(int cc)` |
    `void` | `setDataType​(Types.DataType dt)` | set data type of the output that is produced by this lop
    `void` | `setEndColumn​(int passed)` |
    `void` | `setEndLine​(int passed)` |
    `void` | `setExecType​(Types.ExecType newExecType)` | Set the execution type of LOP.
    `void` | `setFederatedOutput​(FEDInstruction.FederatedOutput fedOutput)` |
    `void` | `setFilename​(String passed)` |
    `void` | `setPrivacyConstraint​(PrivacyConstraint privacy)` | Method to set privacy constraint of Lop.
    `void` | `setValueType​(Types.ValueType vt)` | set value type of the output that is produced by this lop
    `void` | `setVisited​(Lop.VisitStatus visited)` | set visit status of node
    `abstract String` | `toString()` | Method to have Lops print their state.

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, wait, wait, wait`

- - ### Field Detail

    - #### FILE_SEPARATOR

          public static final String FILE_SEPARATOR

      See Also:
        Constant Field Values

    - #### PROCESS_PREFIX

          public static final String PROCESS_PREFIX

      See Also:
        Constant Field Values

    - #### CP_ROOT_THREAD_ID

          public static final String CP_ROOT_THREAD_ID

      See Also:
        Constant Field Values

    - #### CP_CHILD_THREAD

          public static final String CP_CHILD_THREAD

      See Also:
        Constant Field Values

    - #### SAMPLE_FRACTION

          public static final double SAMPLE_FRACTION

      See Also:
        Constant Field Values

    - #### INSTRUCTION_DELIMITOR

          public static final String INSTRUCTION_DELIMITOR

      See Also:
        Constant Field Values

    - #### OPERAND_DELIMITOR

          public static final String OPERAND_DELIMITOR

      See Also:
        Constant Field Values

    - #### VALUETYPE_PREFIX

          public static final String VALUETYPE_PREFIX

      See Also:
        Constant Field Values

    - #### DATATYPE_PREFIX

          public static final String DATATYPE_PREFIX

      See Also:
        Constant Field Values

    - #### LITERAL_PREFIX

          public static final String LITERAL_PREFIX

      See Also:
        Constant Field Values

    - #### VARIABLE_NAME_PLACEHOLDER

          public static final String VARIABLE_NAME_PLACEHOLDER

      See Also:
        Constant Field Values

    - #### NAME_VALUE_SEPARATOR

          public static final String NAME_VALUE_SEPARATOR

      See Also:
        Constant Field Values

    - #### MATRIX_VAR_NAME_PREFIX

          public static final String MATRIX_VAR_NAME_PREFIX

      See Also:
        Constant Field Values

    - #### FRAME_VAR_NAME_PREFIX

          public static final String FRAME_VAR_NAME_PREFIX

      See Also:
        Constant Field Values

    - #### SCALAR_VAR_NAME_PREFIX

          public static final String SCALAR_VAR_NAME_PREFIX

      See Also:
        Constant Field Values

    - #### UPDATE_INPLACE_PREFIX

          public static final String UPDATE_INPLACE_PREFIX

      See Also:
        Constant Field Values

    - #### _beginLine

          public int _beginLine

    - #### _beginColumn

          public int _beginColumn

    - #### _endLine

          public int _endLine

    - #### _endColumn

          public int _endColumn

    - #### _filename

          public String _filename

  - ### Constructor Detail

    - #### Lop

          public Lop​(Lop.Type t,
                     Types.DataType dt,
                     Types.ValueType vt)

      Constructor to be invoked by base class.
      Parameters:
        `t` - lop type
        `dt` - data type of the output
        `vt` - value type of the output

  - ### Method Detail

    - #### getVisited

          public Lop.VisitStatus getVisited()

      get visit status of node
      Returns:
        visit status

    - #### setVisited

          public void setVisited​(Lop.VisitStatus visited)

      set visit status of node
      Parameters:
        `visited` - visit status

    - #### getReachable

          public boolean[] getReachable()

    - #### createReachable

          public boolean[] createReachable​(int size)

    - #### isDataExecLocation

          public boolean isDataExecLocation()

    - #### getDataType

          public Types.DataType getDataType()

      get data type of the output that is produced by this lop
      Returns:
        data type

    - #### setDataType

          public void setDataType​(Types.DataType dt)

      set data type of the output that is produced by this lop
      Parameters:
        `dt` - data type

    - #### getValueType

          public Types.ValueType getValueType()

      get value type of the output that is produced by this lop
      Returns:
        value type

    - #### setValueType

          public void setValueType​(Types.ValueType vt)

      set value type of the output that is produced by this lop
      Parameters:
        `vt` - value type

    - #### getType

          public Lop.Type getType()

      Method to get Lop type.
      Returns:
        lop type

    - #### getInputs

          public ArrayList<Lop> getInputs()

      Method to get input of Lops
      Returns:
        list of input lops

    - #### getOutputs

          public ArrayList<Lop> getOutputs()

      Method to get output of Lops
      Returns:
        list of output lops

    - #### addInput

          public void addInput​(Lop op)

      Method to add input to Lop
      Parameters:
        `op` - input lop

    - #### replaceInput

          public void replaceInput​(Lop oldInp,
                                   Lop newInp)

      Method to replace an input to a Lop
      Parameters:
        `oldInp` - old input Lop
        `newInp` - new input Lop

    - #### addOutput

          public void addOutput​(Lop op)

      Method to add output to Lop
      Parameters:
        `op` - output lop

    - #### removeOutput

          public void removeOutput​(Lop op)

      Method to remove output from Lop
      Parameters:
        `op` - Lop to remove

    - #### setPrivacyConstraint

          public void setPrivacyConstraint​(PrivacyConstraint privacy)

      Method to set privacy constraint of Lop.
      Parameters:
        `privacy` - privacy constraint instance

    - #### getPrivacyConstraint

          public PrivacyConstraint getPrivacyConstraint()

    - #### activatePrefetch

          public void activatePrefetch()

    - #### prefetchActivated

          public boolean prefetchActivated()

    - #### setFederatedOutput

          public void setFederatedOutput​(FEDInstruction.FederatedOutput fedOutput)

    - #### getFederatedOutput

          public FEDInstruction.FederatedOutput getFederatedOutput()

    - #### setConsumerCount

          public void setConsumerCount​(int cc)

    - #### removeConsumer

          public int removeConsumer()

    - #### toString

          public abstract String toString()

      Method to have Lops print their state. This is for debugging purposes.
      Overrides:
        `toString` in class `Object`

    - #### resetVisitStatus

          public void resetVisitStatus()

    - #### getID

          public long getID()

      Method to return the ID of LOP
      Returns:
        lop ID

    - #### getLevel

          public int getLevel()

    - #### getExecType

          public Types.ExecType getExecType()

      Method to get the execution type (CP, CP_FILE, MR, SPARK, GPU, FED, INVALID) of LOP
      Returns:
        execution type

    - #### setExecType

          public void setExecType​(Types.ExecType newExecType)

      Set the execution type of LOP.
      Parameters:
        `newExecType` - new execution type

    - #### getProducesIntermediateOutput

          public boolean getProducesIntermediateOutput()

    - #### addToDag

          public final void addToDag​(Dag<Lop> dag)

      Method to recursively add LOPS to a DAG
      Parameters:
        `dag` - lop DAG

    - #### getOutputParameters

          public OutputParameters getOutputParameters()

      Method to get output parameters
      Returns:
        output parameters

    - #### getAggType

          public AggBinaryOp.SparkAggType getAggType()

      Method to get aggregate type if applicable. This method is overridden by the Lops with aggregate types (e.g. MapMult)
      Returns:
        SparkAggType

    - #### getBroadcastInput

          public Lop getBroadcastInput()

      Method to get the input to be broadcast. This method is overridden by the Lops which require broadcasts (e.g. AppendM)
      Returns:
        An input Lop or Null

    - #### getInstructions

          public String getInstructions​(String output)

      Method should be overridden if needed
      Parameters:
        `output` - output
      Returns:
        instructions as string

    - #### getInstructions

          public String getInstructions​(String input1,
                                        String output)

      Method should be overridden if needed
      Parameters:
        `input1` - input 1
        `output` - output
      Returns:
        instructions as string

    - #### getInstructions

          public String getInstructions​(String input1,
                                        String input2,
                                        String output)

      Method should be overridden if needed
      Parameters:
        `input1` - input 1
        `input2` - input 2
        `output` - output
      Returns:
        instructions as string

    - #### getInstructions

          public String getInstructions​(String input1,
                                        String input2,
                                        String input3,
                                        String output)

      Method should be overridden if needed
      Parameters:
        `input1` - input 1
        `input2` - input 2
        `input3` - input 3
        `output` - output
      Returns:
        instructions as string

    - #### getInstructions

          public String getInstructions​(String input1,
                                        String input2,
                                        String input3,
                                        String input4,
                                        String output)

      Method should be overridden if needed
      Parameters:
        `input1` - input 1
        `input2` - input 2
        `input3` - input 3
        `input4` - input 4
        `output` - output
      Returns:
        instructions as string

    - #### getInstructions

          public String getInstructions​(String input1,
                                        String input2,
                                        String input3,
                                        String input4,
                                        String input5,
                                        String output)

      Method should be overridden if needed
      Parameters:
        `input1` - input 1
        `input2` - input 2
        `input3` - input 3
        `input4` - input 4
        `input5` - input 5
        `output` - output
      Returns:
        instructions as string

    - #### getInstructions

          public String getInstructions​(String input1,
                                        String input2,
                                        String input3,
                                        String input4,
                                        String input5,
                                        String input6,
                                        String output)

      Method should be overridden if needed
      Parameters:
        `input1` - input 1
        `input2` - input 2
        `input3` - input 3
        `input4` - input 4
        `input5` - input 5
        `input6` - input 6
        `output` - output
      Returns:
        instructions as string

    - #### getInstructions

          public String getInstructions​(String input1,
                                        String input2,
                                        String input3,
                                        String input4,
                                        String input5,
                                        String input6,
                                        String input7,
                                        String output)

    - #### getInstructions

          public String getInstructions​(String[] inputs,
                                        String outputs)

    - #### getInstructions

          public String getInstructions​(String[] inputs,
                                        String[] outputs)

      Method should be overridden if needed
      Parameters:
        `inputs` - array of inputs
        `outputs` - array of outputs
      Returns:
        instructions as string

    - #### getInstructions

          public String getInstructions()

      Method should be overridden if needed
      Returns:
        instructions as string

    - #### getSimpleInstructionType

          public Lop.SimpleInstType getSimpleInstructionType()

      Method should be overridden if needed
      Returns:
        simple instruction type

    - #### setBeginLine

          public void setBeginLine​(int passed)

    - #### setBeginColumn

          public void setBeginColumn​(int passed)

    - #### setEndLine

          public void setEndLine​(int passed)

    - #### setEndColumn

          public void setEndColumn​(int passed)

    - #### setFilename

          public void setFilename​(String passed)

    - #### setAllPositions

          public void setAllPositions​(String filename,
                                      int blp,
                                      int bcp,
                                      int elp,
                                      int ecp)

    - #### getBeginLine

          public int getBeginLine()

    - #### getBeginColumn

          public int getBeginColumn()

    - #### getEndLine

          public int getEndLine()

    - #### getEndColumn

          public int getEndColumn()

    - #### getFilename

          public String getFilename()

    - #### printErrorLocation

          public String printErrorLocation()

    - #### getInstructions

          public String getInstructions​(int input,
                                        int rowl,
                                        int rowu,
                                        int coll,
                                        int colu,
                                        int leftRowDim,
                                        int leftColDim,
                                        int output)

    - #### isVariable

          public boolean isVariable()

      Function that determines if the output of a LOP is defined by a variable or not.
      Returns:
        true if lop output defined by a variable

    - #### isAllOutputsCP

          public boolean isAllOutputsCP()

      Function that determines if all the outputs of a LOP are of CP execution types
      Returns:
        true if all outputs are CP

    - #### prepOperand

          public String prepOperand​(String label,
                                    Types.DataType dt,
                                    Types.ValueType vt)

      Method to prepare instruction operand with given parameters.
      Parameters:
        `label` - instruction label
        `dt` - data type
        `vt` - value type
      Returns:
        instruction operand with data type and value type

    - #### prepOperand

          public String prepOperand​(String label,
                                    Types.DataType dt,
                                    Types.ValueType vt,
                                    boolean literal)

      Method to prepare instruction operand with given parameters.
      Parameters:
        `label` - instruction label
        `dt` - data type
        `vt` - value type
        `literal` - true if literal
      Returns:
        instruction operand with data type, value type, and literal status

    - #### prepOutputOperand

          public String prepOutputOperand()

    - #### prepOutputOperand

          public String prepOutputOperand​(int index)

    - #### prepOutputOperand

          public String prepOutputOperand​(String label)

    - #### prepScalarLabel

          public String prepScalarLabel()

      Function to prepare label for scalar inputs while generating instructions. It attaches placeholder suffix and prefixes if the Lop denotes a variable.
      Returns:
        prepared scalar label

    - #### prepScalarOperand

          public String prepScalarOperand​(Types.ExecType et,
                                          String label)

      Function to be used in creating instructions for creating scalar operands. It decides whether or not attach placeholders for instruction patching. Resulting string also encodes if the operand is a literal. For non-literals: Placeholder prefix and suffix need to be attached for Instruction Patching during execution. However, they should NOT be attached IF: - the operand is a literal OR - the execution type is CP. This is because CP runtime has access to symbol table and the instruction encodes sufficient information to determine if an operand is a literal or not.
      Parameters:
        `et` - execution type
        `label` - instruction label
      Returns:
        prepared scalar operand

    - #### prepScalarInputOperand

          public String prepScalarInputOperand​(Types.ExecType et)

    - #### prepScalarInputOperand

          public String prepScalarInputOperand​(String label)

    - #### prepInputOperand

          public String prepInputOperand​(int index)

    - #### prepInputOperand

          public String prepInputOperand​(String label)