org.apache.sysds.runtime.instructions.spark

## Class MatrixIndexingSPInstruction

- java.lang.Object
- - org.apache.sysds.runtime.instructions.Instruction
  - - org.apache.sysds.runtime.instructions.spark.SPInstruction
    - - org.apache.sysds.runtime.instructions.spark.ComputationSPInstruction
      - - org.apache.sysds.runtime.instructions.spark.UnarySPInstruction
        - - org.apache.sysds.runtime.instructions.spark.IndexingSPInstruction
          - - org.apache.sysds.runtime.instructions.spark.MatrixIndexingSPInstruction

- All Implemented Interfaces:
    LineageTraceable

      public class MatrixIndexingSPInstruction
      extends IndexingSPInstruction

  This class implements the matrix indexing functionality inside CP.

- - ### Nested Class Summary

    - ### Nested classes/interfaces inherited from class org.apache.sysds.runtime.instructions.spark.SPInstruction

      `SPInstruction.SPType`

    - ### Nested classes/interfaces inherited from class org.apache.sysds.runtime.instructions.Instruction

      `Instruction.IType`

  - ### Field Summary

    - ### Fields inherited from class org.apache.sysds.runtime.instructions.spark.ComputationSPInstruction

      `input1, input2, input3, output`

    - ### Fields inherited from class org.apache.sysds.runtime.instructions.Instruction

      `DATATYPE_PREFIX, FEDERATED_INST_PREFIX, GPU_INST_PREFIX, INSTRUCTION_DELIM, LITERAL_PREFIX, OPERAND_DELIM, SP_INST_PREFIX, VALUETYPE_PREFIX`

  - ### Method Summary

    All Methods Static Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `org.apache.commons.lang3.tuple.Pair<String,LineageItem>` | `getLineageItem(ExecutionContext ec)` Obtain lineage trace of an instruction with a single output.
    `static MatrixBlock` | `inmemoryIndexing(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in1, DataCharacteristics mcIn, DataCharacteristics mcOut, IndexRange ixrange)`
    `static boolean` | `isMultiBlockLookup(org.apache.spark.api.java.JavaPairRDD<?,?> in, DataCharacteristics mcIn, DataCharacteristics mcOut, IndexRange ixrange)` Indicates if the given index range and input matrix exhibit the following properties: (1) existing hash partitioner, (2) out-of-core input matrix (larger than aggregated memory), (3) aligned indexing range (which does not required aggregation), and (4) the output fits twice in memory (in order to collect the result).
    `static boolean` | `isSingleBlockLookup(DataCharacteristics mcIn, IndexRange ixrange)` Indicates if the given index range only covers a single blocks of the inputs matrix.
    `void` | `processInstruction(ExecutionContext ec)` This method should be used to execute the instruction.

    - ### Methods inherited from class org.apache.sysds.runtime.instructions.spark.IndexingSPInstruction

      `parseInstruction`

    - ### Methods inherited from class org.apache.sysds.runtime.instructions.spark.ComputationSPInstruction

      `getOutputVariableName`

    - ### Methods inherited from class org.apache.sysds.runtime.instructions.spark.SPInstruction

      `getGraphString, getSPInstructionType, getType, postprocessInstruction, preprocessInstruction, requiresLabelUpdate`

    - ### Methods inherited from class org.apache.sysds.runtime.instructions.Instruction

      `getBeginColumn, getBeginLine, getEndColumn, getEndLine, getExtendedOpcode, getFilename, getInstID, getInstructionString, getLineNum, getOpcode, getOperator, getPrivacyConstraint, printMe, setInstID, setLocation, setLocation, setLocation, setLocation, setPrivacyConstraint, setPrivacyConstraint, toString, updateInstructionThreadID`

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, wait, wait, wait`

    - ### Methods inherited from interface org.apache.sysds.runtime.lineage.LineageTraceable

      `getLineageItems, hasSingleLineage`

- - ### Method Detail

    - #### processInstruction

          public void processInstruction(ExecutionContext ec)

      Description copied from class: `Instruction`
      This method should be used to execute the instruction.
      Specified by:
        `processInstruction` in class `SPInstruction`
      Parameters:
        `ec` - execution context

    - #### inmemoryIndexing

          public static MatrixBlock inmemoryIndexing(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in1,
                                                     DataCharacteristics mcIn,
                                                     DataCharacteristics mcOut,
                                                     IndexRange ixrange)

    - #### isSingleBlockLookup

          public static boolean isSingleBlockLookup(DataCharacteristics mcIn,
                                                    IndexRange ixrange)

      Indicates if the given index range only covers a single blocks of the inputs matrix. In this case, we perform a key lookup which is very efficient in case of existing partitioner, especially for out-of-core datasets.
      Parameters:
        `mcIn` - matrix characteristics
        `ixrange` - index range
      Returns:
        true if index range covers a single block of the input matrix

    - #### isMultiBlockLookup

          public static boolean isMultiBlockLookup(org.apache.spark.api.java.JavaPairRDD<?,?> in,
                                                   DataCharacteristics mcIn,
                                                   DataCharacteristics mcOut,
                                                   IndexRange ixrange)

      Indicates if the given index range and input matrix exhibit the following properties: (1) existing hash partitioner, (2) out-of-core input matrix (larger than aggregated memory), (3) aligned indexing range (which does not required aggregation), and (4) the output fits twice in memory (in order to collect the result).
      Parameters:
        `in` - input matrix
        `mcIn` - input matrix characteristics
        `mcOut` - output matrix characteristics
        `ixrange` - index range
      Returns:
        true if index range requires a multi-block lookup

    - #### getLineageItem

          public org.apache.commons.lang3.tuple.Pair<String,LineageItem> getLineageItem(ExecutionContext ec)

      Description copied from interface: `LineageTraceable`
      Obtain lineage trace of an instruction with a single output.
      Specified by:
        `getLineageItem` in interface `LineageTraceable`
      Overrides:
        `getLineageItem` in class `ComputationSPInstruction`
      Parameters:
        `ec` - execution context w/ live variables
      Returns:
        pair of (output variable name, output lineage item)

Copyright © 2021 The Apache Software Foundation. All rights reserved.