org.apache.sysds.hops

## Class AggBinaryOp

- java.lang.Object
- - org.apache.sysds.hops.Hop
  - - org.apache.sysds.hops.MultiThreadedHop
    - - org.apache.sysds.hops.AggBinaryOp

- All Implemented Interfaces:
    ParseInfo

      public class AggBinaryOp
      extends MultiThreadedHop

- - ### Nested Class Summary

    Nested Classes
    Modifier and Type | Class and Description
    --- | ---
    `static class` | `AggBinaryOp.MMultMethod`
    `static class` | `AggBinaryOp.SparkAggType`

  - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static AggBinaryOp.MMultMethod` | `FORCED_MMULT_METHOD`
    `static double` | `MAPMULT_MEM_MULTIPLIER`

    - ### Fields inherited from class org.apache.sysds.hops.Hop

      `_beginColumn, _beginLine, _endColumn, _endLine, _filename, _text, CPThreshold`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `AggBinaryOp(String l, Types.DataType dt, Types.ValueType vt, Types.OpOp2 innOp, Types.AggOp outOp, Hop in1, Hop in2)`

  - ### Method Summary

    All Methods Static Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `boolean` | `allowsAllExecTypes()`
    `void` | `checkArity()` Check whether this Hop has a correct number of inputs.
    `MapMultChain.ChainType` | `checkMapMultChain()` MapMultChain: Determine if XtwXv/XtXv pattern applies for this aggbinary and if yes which type.
    `MMTSJ.MMTSJType` | `checkTransposeSelf()` TSMM: Determine if XtX pattern applies for this aggbinary and if yes which type.
    `Object` | `clone()`
    `boolean` | `compare(Hop that)`
    `void` | `computeMemEstimate(MemoTable memo)` Computes the estimate of memory required to store the input/output of this hop in memory.
    `Lop` | `constructLops()` NOTE: overestimated mem in case of transpose-identity matmult, but 3/2 at worst and existing mem estimate advantageous in terms of consistency hops/lops, and some special cases internally materialize the transpose for better cache locality
    `static double` | `getMapmmMemEstimate(long m1_rows, long m1_cols, long m1_blen, long m1_nnz, long m2_rows, long m2_cols, long m2_blen, long m2_nnz, int cachedInputIndex, boolean pmm)` Estimates the memory footprint of MapMult operation depending on which input is put into distributed cache.
    `AggBinaryOp.MMultMethod` | `getMMultMethod()`
    `String` | `getOpString()`
    `boolean` | `hasLeftPMInput()`
    `boolean` | `isGPUEnabled()` In memory-based optimizer mode (see OptimizerUtils.isMemoryBasedOptLevel()), the exectype is determined by checking this method as well as memory budget of this Hop.
    `boolean` | `isMatrixMultiply()`
    `boolean` | `isMultiThreadedOpType()`
    `void` | `refreshSizeInformation()` Update the output size information for this hop.
    `void` | `setHasLeftPMInput(boolean flag)`

    - ### Methods inherited from class org.apache.sysds.hops.MultiThreadedHop

      `getMaxNumThreads, setMaxNumThreads`

    - ### Methods inherited from class org.apache.sysds.hops.Hop

      `addAllInputs, addInput, checkAndSetForcedPlatform, checkAndSetInvalidCPDimsAndSize, clearMemEstimate, colsKnown, compressedSize, computeBoundsInformation, computeBoundsInformation, computeBoundsInformation, computeSizeInformation, computeSizeInformation, computeSizeInformation, constructAndSetLopsDataFlowProperties, createOffsetLop, dimsKnown, dimsKnown, dimsKnownAny, getBeginColumn, getBeginLine, getBlocksize, getCompressedSize, getDataCharacteristics, getDataType, getDim, getDim1, getDim2, getEndColumn, getEndLine, getExecType, getFilename, getForcedExecType, getHopID, getInput, getInput, getInputMemEstimate, getInputOutputSize, getIntermediateMemEstimate, getLength, getLops, getMemEstimate, getName, getNnz, getOutputMemEstimate, getParent, getPrivacy, getSparsity, getSpBroadcastSize, getText, getUpdateType, getValueType, hasCompressedInput, hasFederatedOutput, hasMatrixInputWithDifferentBlocksizes, hasValidCPDimsAndSize, isCompressedOutput, isFederated, isFederatedOutput, isMatrix, isMemEstimated, isOutputEmptyBlocks, isRequiredDecompression, isScalar, isTransposeSafe, isVisited, printErrorLocation, refreshColsParameterInformation, refreshColsParameterInformation, refreshMemEstimates, refreshRowsParameterInformation, refreshRowsParameterInformation, requiresCheckpoint, requiresCompression, requiresLineageCaching, requiresReblock, requiresRecompile, resetExecType, resetRecompilationFlag, resetRecompilationFlag, resetVisitStatus, resetVisitStatus, resetVisitStatus, resetVisitStatusForced, rowsKnown, setBeginColumn, setBeginLine, setBlocksize, setCompressedOutput, setCompressedSize, setDataType, setDim, setDim1, setDim2, setEndColumn, setEndLine, setExecType, setFederatedOutput, setFilename, setForcedExecType, setLops, setMemEstimate, setName, setNnz, setOutputEmptyBlocks, setParseInfo, setPrivacy, setRequiresCheckpoint, setRequiresCompression, setRequiresCompression, setRequiresDeCompression, setRequiresLineageCaching, setRequiresReblock, setRequiresRecompile, setText, setUpdateType, setValueType, setVisited, setVisited, someInputFederated, toString`

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, wait, wait, wait`

    - ### Methods inherited from interface org.apache.sysds.parser.ParseInfo

      `ctxAndFilenameToParseInfo`

- - ### Field Detail

    - #### MAPMULT_MEM_MULTIPLIER

          public static final double MAPMULT_MEM_MULTIPLIER

      See Also:
        Constant Field Values

    - #### FORCED_MMULT_METHOD

          public static AggBinaryOp.MMultMethod FORCED_MMULT_METHOD

  - ### Constructor Detail

    - #### AggBinaryOp

          public AggBinaryOp(String l,
                             Types.DataType dt,
                             Types.ValueType vt,
                             Types.OpOp2 innOp,
                             Types.AggOp outOp,
                             Hop in1,
                             Hop in2)

  - ### Method Detail

    - #### checkArity

          public void checkArity()

      Description copied from class: `Hop`
      Check whether this Hop has a correct number of inputs. (Some Hops can have a variable number of inputs, such as DataOp, DataGenOp, ParameterizedBuiltinOp, ReorgOp, TernaryOp, QuaternaryOp, MultipleOp, DnnOp, and SpoofFusedOp.) Parameterized Hops (such as DataOp) can check that the number of parameters matches the number of inputs.
      Specified by:
        `checkArity` in class `Hop`

    - #### setHasLeftPMInput

          public void setHasLeftPMInput(boolean flag)

    - #### hasLeftPMInput

          public boolean hasLeftPMInput()

    - #### getMMultMethod

          public AggBinaryOp.MMultMethod getMMultMethod()

    - #### isGPUEnabled

          public boolean isGPUEnabled()

      Description copied from class: `Hop`
      In memory-based optimizer mode (see OptimizerUtils.isMemoryBasedOptLevel()), the exectype is determined by checking this method as well as memory budget of this Hop. Please see findExecTypeByMemEstimate for more detail. This method is necessary because not all operator are supported efficiently on GPU (for example: operations on frames and scalar as well as operations such as table).
      Specified by:
        `isGPUEnabled` in class `Hop`
      Returns:
        true if the Hop is eligible for GPU Exectype.

    - #### constructLops

          public Lop constructLops()

      NOTE: overestimated mem in case of transpose-identity matmult, but 3/2 at worst and existing mem estimate advantageous in terms of consistency hops/lops, and some special cases internally materialize the transpose for better cache locality
      Specified by:
        `constructLops` in class `Hop`

    - #### getOpString

          public String getOpString()

      Specified by:
        `getOpString` in class `Hop`

    - #### computeMemEstimate

          public void computeMemEstimate(MemoTable memo)

      Description copied from class: `Hop`
      Computes the estimate of memory required to store the input/output of this hop in memory. This is the default implementation (orchestration of hop-specific implementation) that should suffice for most hops. If a hop requires more control, this method should be overwritten with awareness of (1) output estimates, and (2) propagation of worst-case matrix characteristics (dimensions, sparsity). TODO remove memo table and, on constructor refresh, inference in refresh, single compute mem, maybe general computeMemEstimate, flags to indicate if estimate or not.
      Overrides:
        `computeMemEstimate` in class `Hop`
      Parameters:
        `memo` - memory table

    - #### isMatrixMultiply

          public boolean isMatrixMultiply()

    - #### isMultiThreadedOpType

          public boolean isMultiThreadedOpType()

      Specified by:
        `isMultiThreadedOpType` in class `MultiThreadedHop`

    - #### allowsAllExecTypes

          public boolean allowsAllExecTypes()

      Specified by:
        `allowsAllExecTypes` in class `Hop`

    - #### checkTransposeSelf

          public MMTSJ.MMTSJType checkTransposeSelf()

      TSMM: Determine if XtX pattern applies for this aggbinary and if yes which type.
      Returns:
        MMTSJType

    - #### checkMapMultChain

          public MapMultChain.ChainType checkMapMultChain()

      MapMultChain: Determine if XtwXv/XtXv pattern applies for this aggbinary and if yes which type.
      Returns:
        ChainType

    - #### getMapmmMemEstimate

          public static double getMapmmMemEstimate(long m1_rows,
                                                   long m1_cols,
                                                   long m1_blen,
                                                   long m1_nnz,
                                                   long m2_rows,
                                                   long m2_cols,
                                                   long m2_blen,
                                                   long m2_nnz,
                                                   int cachedInputIndex,
                                                   boolean pmm)

      Estimates the memory footprint of MapMult operation depending on which input is put into distributed cache. This function is called by `optFindMMultMethod()` to decide the execution strategy, as well as by piggybacking to decide the number of Map-side instructions to put into a single GMR job.
      Parameters:
        `m1_rows` - m1 rows
        `m1_cols` - m1 cols
        `m1_blen` - m1 rows/cols per block
        `m1_nnz` - m1 num non-zeros
        `m2_rows` - m2 rows
        `m2_cols` - m2 cols
        `m2_blen` - m2 rows/cols per block
        `m2_nnz` - m2 num non-zeros
        `cachedInputIndex` - true if cached input index
        `pmm` - true if permutation matrix multiply
      Returns:
        map mm memory estimate

    - #### refreshSizeInformation

          public void refreshSizeInformation()

      Description copied from class: `Hop`
      Update the output size information for this hop.
      Specified by:
        `refreshSizeInformation` in class `Hop`

    - #### clone

          public Object clone()
                       throws CloneNotSupportedException

      Specified by:
        `clone` in class `Hop`
      Throws:
        `CloneNotSupportedException`

    - #### compare

          public boolean compare(Hop that)

      Specified by:
        `compare` in class `Hop`

Copyright © 2021 The Apache Software Foundation. All rights reserved.