org.apache.sysds.hops

## Class Hop

- java.lang.Object
- - org.apache.sysds.hops.Hop

- All Implemented Interfaces:
    ParseInfo
  Direct Known Subclasses:
    DataOp, FunctionOp, IndexingOp, LeftIndexingOp, LiteralOp, MultiThreadedHop, NaryOp

      public abstract class Hop
      extends Object
      implements ParseInfo

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `int` | `_beginColumn`
    `int` | `_beginLine`
    `int` | `_endColumn`
    `int` | `_endLine`
    `String` | `_filename`
    `String` | `_text`
    `static long` | `CPThreshold`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `Hop(String l, Types.DataType dt, Types.ValueType vt)`

  - ### Method Summary

    All Methods Static Methods Instance Methods Abstract Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `void` | `addAllInputs(ArrayList<Hop> list)`
    `void` | `addInput(Hop h)`
    `abstract boolean` | `allowsAllExecTypes()`
    `void` | `checkAndSetForcedPlatform()`
    `void` | `checkAndSetInvalidCPDimsAndSize()`
    `abstract void` | `checkArity()` Check whether this Hop has a correct number of inputs.
    `void` | `clearMemEstimate()`
    `abstract Object` | `clone()`
    `boolean` | `colsKnown()`
    `abstract boolean` | `compare(Hop that)`
    `long` | `compressedSize()`
    `double` | `computeBoundsInformation(Hop input)`
    `static double` | `computeBoundsInformation(Hop input, LocalVariableMap vars)`
    `static double` | `computeBoundsInformation(Hop input, LocalVariableMap vars, HashMap<Long,Double> memo)`
    `void` | `computeMemEstimate(MemoTable memo)` Computes the estimate of memory required to store the input/output of this hop in memory.
    `static long` | `computeSizeInformation(Hop input)`
    `long` | `computeSizeInformation(Hop input, LocalVariableMap vars)`
    `long` | `computeSizeInformation(Hop input, LocalVariableMap vars, HashMap<Long,Long> memo)`
    `void` | `constructAndSetLopsDataFlowProperties()`
    `abstract Lop` | `constructLops()`
    `static Lop` | `createOffsetLop(Hop hop, boolean repCols)`
    `boolean` | `dimsKnown()`
    `boolean` | `dimsKnown(boolean includeNnz)`
    `boolean` | `dimsKnownAny()`
    `int` | `getBeginColumn()`
    `int` | `getBeginLine()`
    `int` | `getBlocksize()`
    `long` | `getCompressedSize()`
    `DataCharacteristics` | `getDataCharacteristics()`
    `Types.DataType` | `getDataType()`
    `long` | `getDim(int i)`
    `long` | `getDim1()`
    `long` | `getDim2()`
    `int` | `getEndColumn()`
    `int` | `getEndLine()`
    `Types.ExecType` | `getExecType()`
    `String` | `getFilename()`
    `Types.ExecType` | `getForcedExecType()`
    `long` | `getHopID()`
    `ArrayList<Hop>` | `getInput()`
    `Hop` | `getInput(int ix)`
    `double` | `getInputMemEstimate()`
    `double` | `getInputOutputSize(Collection<String> exclVars)`
    `double` | `getIntermediateMemEstimate()`
    `long` | `getLength()`
    `Lop` | `getLops()`
    `double` | `getMemEstimate()` NOTES: * Purpose: Whenever the output dimensions / sparsity of a hop are unknown, this hop should store its worst-case output statistics (if known) in that table.
    `String` | `getName()`
    `long` | `getNnz()`
    `abstract String` | `getOpString()`
    `double` | `getOutputMemEstimate()`
    `ArrayList<Hop>` | `getParent()`
    `PrivacyConstraint` | `getPrivacy()`
    `double` | `getSparsity()`
    `double` | `getSpBroadcastSize()`
    `String` | `getText()`
    `MatrixObject.UpdateType` | `getUpdateType()`
    `Types.ValueType` | `getValueType()`
    `boolean` | `hasCompressedInput()`
    `boolean` | `hasFederatedOutput()`
    `boolean` | `hasMatrixInputWithDifferentBlocksizes()`
    `boolean` | `hasValidCPDimsAndSize()`
    `boolean` | `isCompressedOutput()`
    `boolean` | `isFederated()`
    `boolean` | `isFederatedOutput()`
    `abstract boolean` | `isGPUEnabled()` In memory-based optimizer mode (see OptimizerUtils.isMemoryBasedOptLevel()), the exectype is determined by checking this method as well as memory budget of this Hop.
    `boolean` | `isMatrix()`
    `boolean` | `isMemEstimated()`
    `boolean` | `isOutputEmptyBlocks()`
    `boolean` | `isRequiredDecompression()`
    `boolean` | `isScalar()`
    `boolean` | `isTransposeSafe()` Defines if this operation is transpose-safe, which means that the result of op(input) is equivalent to op(t(input)).
    `boolean` | `isVisited()`
    `String` | `printErrorLocation()`
    `void` | `refreshColsParameterInformation(Hop input, LocalVariableMap vars)`
    `void` | `refreshColsParameterInformation(Hop input, LocalVariableMap vars, HashMap<Long,Long> memo)`
    `void` | `refreshMemEstimates(MemoTable memo)` Recursively computes memory estimates for all the Hops in the DAG rooted at the current hop pointed by `this`.
    `void` | `refreshRowsParameterInformation(Hop input, LocalVariableMap vars)`
    `void` | `refreshRowsParameterInformation(Hop input, LocalVariableMap vars, HashMap<Long,Long> memo)`
    `abstract void` | `refreshSizeInformation()` Update the output size information for this hop.
    `boolean` | `requiresCheckpoint()`
    `boolean` | `requiresCompression()`
    `boolean` | `requiresLineageCaching()`
    `boolean` | `requiresReblock()`
    `boolean` | `requiresRecompile()` Indicates if dynamic recompilation is required for this hop.
    `void` | `resetExecType()`
    `static void` | `resetRecompilationFlag(ArrayList<Hop> hops, Types.ExecType et, Recompiler.ResetType reset)`
    `static void` | `resetRecompilationFlag(Hop hops, Types.ExecType et, Recompiler.ResetType reset)`
    `Hop` | `resetVisitStatus()`
    `static void` | `resetVisitStatus(ArrayList<Hop> hops)`
    `static void` | `resetVisitStatus(ArrayList<Hop> hops, boolean force)`
    `void` | `resetVisitStatusForced(HashSet<Long> memo)`
    `boolean` | `rowsKnown()`
    `void` | `setBeginColumn(int passed)`
    `void` | `setBeginLine(int passed)`
    `void` | `setBlocksize(int blen)`
    `void` | `setCompressedOutput(boolean value)`
    `void` | `setCompressedSize(long size)`
    `void` | `setDataType(Types.DataType dt)`
    `void` | `setDim(int i, long dim)`
    `void` | `setDim1(long dim1)`
    `void` | `setDim2(long dim2)`
    `void` | `setEndColumn(int passed)`
    `void` | `setEndLine(int passed)`
    `void` | `setExecType(Types.ExecType execType)`
    `void` | `setFederatedOutput(FEDInstruction.FederatedOutput federatedOutput)`
    `void` | `setFilename(String passed)`
    `void` | `setForcedExecType(Types.ExecType etype)`
    `void` | `setLops(Lop lops)`
    `void` | `setMemEstimate(double mem)` Sets memory estimate in bytes
    `void` | `setName(String _name)`
    `void` | `setNnz(long nnz)`
    `void` | `setOutputEmptyBlocks(boolean flag)`
    `void` | `setParseInfo(ParseInfo parseInfo)` Set parse information.
    `void` | `setPrivacy(PrivacyConstraint privacy)`
    `void` | `setRequiresCheckpoint(boolean flag)`
    `void` | `setRequiresCompression()`
    `void` | `setRequiresCompression(AWTreeNode node)`
    `void` | `setRequiresDeCompression()`
    `void` | `setRequiresLineageCaching(boolean flag)`
    `void` | `setRequiresReblock(boolean flag)`
    `void` | `setRequiresRecompile()` Marks the hop for dynamic recompilation.
    `void` | `setText(String text)`
    `void` | `setUpdateType(MatrixObject.UpdateType update)`
    `void` | `setValueType(Types.ValueType vt)`
    `void` | `setVisited()`
    `void` | `setVisited(boolean flag)`
    `boolean` | `someInputFederated()`
    `String` | `toString()`

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, wait, wait, wait`

    - ### Methods inherited from interface org.apache.sysds.parser.ParseInfo

      `ctxAndFilenameToParseInfo`

- - ### Field Detail

    - #### CPThreshold

          public static final long CPThreshold

      See Also:
        Constant Field Values

    - #### _beginLine

          public int _beginLine

    - #### _beginColumn

          public int _beginColumn

    - #### _endLine

          public int _endLine

    - #### _endColumn

          public int _endColumn

    - #### _filename

          public String _filename

    - #### _text

          public String _text

  - ### Constructor Detail

    - #### Hop

          public Hop(String l,
                     Types.DataType dt,
                     Types.ValueType vt)

  - ### Method Detail

    - #### getHopID

          public long getHopID()

    - #### checkArity

          public abstract void checkArity()

      Check whether this Hop has a correct number of inputs. (Some Hops can have a variable number of inputs, such as DataOp, DataGenOp, ParameterizedBuiltinOp, ReorgOp, TernaryOp, QuaternaryOp, MultipleOp, DnnOp, and SpoofFusedOp.) Parameterized Hops (such as DataOp) can check that the number of parameters matches the number of inputs.

    - #### getExecType

          public Types.ExecType getExecType()

    - #### setExecType

          public void setExecType(Types.ExecType execType)

    - #### setFederatedOutput

          public void setFederatedOutput(FEDInstruction.FederatedOutput federatedOutput)

    - #### resetExecType

          public void resetExecType()

    - #### getForcedExecType

          public Types.ExecType getForcedExecType()

    - #### setForcedExecType

          public void setForcedExecType(Types.ExecType etype)

    - #### allowsAllExecTypes

          public abstract boolean allowsAllExecTypes()

    - #### isTransposeSafe

          public boolean isTransposeSafe()

      Defines if this operation is transpose-safe, which means that the result of op(input) is equivalent to op(t(input)). Usually, this applies to aggregate operations with fixed output dimension. Finally, this information is very useful in order to safely optimize the plan for sparse vectors, which otherwise would be (currently) always represented dense.
      Returns:
        always returns false

    - #### checkAndSetForcedPlatform

          public void checkAndSetForcedPlatform()

    - #### checkAndSetInvalidCPDimsAndSize

          public void checkAndSetInvalidCPDimsAndSize()

    - #### hasValidCPDimsAndSize

          public boolean hasValidCPDimsAndSize()

    - #### hasMatrixInputWithDifferentBlocksizes

          public boolean hasMatrixInputWithDifferentBlocksizes()

    - #### setRequiresReblock

          public void setRequiresReblock(boolean flag)

    - #### requiresReblock

          public boolean requiresReblock()

    - #### setRequiresCheckpoint

          public void setRequiresCheckpoint(boolean flag)

    - #### requiresCheckpoint

          public boolean requiresCheckpoint()

    - #### setRequiresCompression

          public void setRequiresCompression()

    - #### setRequiresCompression

          public void setRequiresCompression(AWTreeNode node)

    - #### setRequiresDeCompression

          public void setRequiresDeCompression()

    - #### isRequiredDecompression

          public boolean isRequiredDecompression()

    - #### requiresCompression

          public boolean requiresCompression()

    - #### setCompressedOutput

          public void setCompressedOutput(boolean value)

    - #### setCompressedSize

          public void setCompressedSize(long size)

    - #### getCompressedSize

          public long getCompressedSize()

    - #### isCompressedOutput

          public boolean isCompressedOutput()

    - #### hasCompressedInput

          public boolean hasCompressedInput()

    - #### compressedSize

          public long compressedSize()

    - #### setRequiresLineageCaching

          public void setRequiresLineageCaching(boolean flag)

    - #### requiresLineageCaching

          public boolean requiresLineageCaching()

    - #### constructAndSetLopsDataFlowProperties

          public void constructAndSetLopsDataFlowProperties()

    - #### createOffsetLop

          public static Lop createOffsetLop(Hop hop,
                                            boolean repCols)

    - #### setOutputEmptyBlocks

          public void setOutputEmptyBlocks(boolean flag)

    - #### isOutputEmptyBlocks

          public boolean isOutputEmptyBlocks()

    - #### getInputOutputSize

          public double getInputOutputSize(Collection<String> exclVars)

    - #### getMemEstimate

          public double getMemEstimate()

      NOTES: * Purpose: Whenever the output dimensions / sparsity of a hop are unknown, this hop should store its worst-case output statistics (if known) in that table. Subsequent hops can then * Invocation: Intended to be called for ALL root nodes of one Hops DAG with the same (initially empty) memo table.
      Returns:
        memory estimate

    - #### setMemEstimate

          public void setMemEstimate(double mem)

      Sets memory estimate in bytes
      Parameters:
        `mem` - memory estimate

    - #### clearMemEstimate

          public void clearMemEstimate()

    - #### isMemEstimated

          public boolean isMemEstimated()

    - #### getInputMemEstimate

          public double getInputMemEstimate()

    - #### getOutputMemEstimate

          public double getOutputMemEstimate()

    - #### getIntermediateMemEstimate

          public double getIntermediateMemEstimate()

    - #### getSpBroadcastSize

          public double getSpBroadcastSize()

    - #### computeMemEstimate

          public void computeMemEstimate(MemoTable memo)

      Computes the estimate of memory required to store the input/output of this hop in memory. This is the default implementation (orchestration of hop-specific implementation) that should suffice for most hops. If a hop requires more control, this method should be overwritten with awareness of (1) output estimates, and (2) propagation of worst-case matrix characteristics (dimensions, sparsity). TODO remove memo table and, on constructor refresh, inference in refresh, single compute mem, maybe general computeMemEstimate, flags to indicate if estimate or not.
      Parameters:
        `memo` - memory table

    - #### refreshMemEstimates

          public void refreshMemEstimates(MemoTable memo)

      Recursively computes memory estimates for all the Hops in the DAG rooted at the current hop pointed by `this`.
      Parameters:
        `memo` - memory table

    - #### isFederated

          public boolean isFederated()

    - #### isFederatedOutput

          public boolean isFederatedOutput()

    - #### someInputFederated

          public boolean someInputFederated()

    - #### getParent

          public ArrayList<Hop> getParent()

    - #### getInput

          public ArrayList<Hop> getInput()

    - #### getInput

          public Hop getInput(int ix)

    - #### addInput

          public void addInput(Hop h)

    - #### addAllInputs

          public void addAllInputs(ArrayList<Hop> list)

    - #### getBlocksize

          public int getBlocksize()

    - #### setBlocksize

          public void setBlocksize(int blen)

    - #### setNnz

          public void setNnz(long nnz)

    - #### getNnz

          public long getNnz()

    - #### setPrivacy

          public void setPrivacy(PrivacyConstraint privacy)

    - #### getPrivacy

          public PrivacyConstraint getPrivacy()

    - #### hasFederatedOutput

          public boolean hasFederatedOutput()

    - #### setUpdateType

          public void setUpdateType(MatrixObject.UpdateType update)

    - #### getUpdateType

          public MatrixObject.UpdateType getUpdateType()

    - #### constructLops

          public abstract Lop constructLops()

    - #### getOpString

          public abstract String getOpString()

    - #### toString

          public String toString()

      Overrides:
        `toString` in class `Object`

    - #### isGPUEnabled

          public abstract boolean isGPUEnabled()

      In memory-based optimizer mode (see OptimizerUtils.isMemoryBasedOptLevel()), the exectype is determined by checking this method as well as memory budget of this Hop. Please see findExecTypeByMemEstimate for more detail. This method is necessary because not all operator are supported efficiently on GPU (for example: operations on frames and scalar as well as operations such as table).
      Returns:
        true if the Hop is eligible for GPU Exectype.

    - #### dimsKnown

          public boolean dimsKnown()

    - #### dimsKnown

          public boolean dimsKnown(boolean includeNnz)

    - #### dimsKnownAny

          public boolean dimsKnownAny()

    - #### rowsKnown

          public boolean rowsKnown()

    - #### colsKnown

          public boolean colsKnown()

    - #### resetVisitStatus

          public static void resetVisitStatus(ArrayList<Hop> hops)

    - #### resetVisitStatus

          public static void resetVisitStatus(ArrayList<Hop> hops,
                                              boolean force)

    - #### resetVisitStatus

          public Hop resetVisitStatus()

    - #### resetVisitStatusForced

          public void resetVisitStatusForced(HashSet<Long> memo)

    - #### resetRecompilationFlag

          public static void resetRecompilationFlag(ArrayList<Hop> hops,
                                                    Types.ExecType et,
                                                    Recompiler.ResetType reset)

    - #### resetRecompilationFlag

          public static void resetRecompilationFlag(Hop hops,
                                                    Types.ExecType et,
                                                    Recompiler.ResetType reset)

    - #### getDim1

          public long getDim1()

    - #### setDim1

          public void setDim1(long dim1)

    - #### getDim2

          public long getDim2()

    - #### setDim2

          public void setDim2(long dim2)

    - #### getDim

          public long getDim(int i)

    - #### setDim

          public void setDim(int i,
                             long dim)

    - #### getLength

          public long getLength()

    - #### getSparsity

          public double getSparsity()

    - #### getDataCharacteristics

          public DataCharacteristics getDataCharacteristics()

    - #### getLops

          public Lop getLops()

    - #### setLops

          public void setLops(Lop lops)

    - #### isVisited

          public boolean isVisited()

    - #### getDataType

          public Types.DataType getDataType()

    - #### setDataType

          public void setDataType(Types.DataType dt)

    - #### isScalar

          public boolean isScalar()

    - #### isMatrix

          public boolean isMatrix()

    - #### setVisited

          public void setVisited()

    - #### setVisited

          public void setVisited(boolean flag)

    - #### setName

          public void setName(String _name)

    - #### getName

          public String getName()

    - #### getValueType

          public Types.ValueType getValueType()

    - #### setValueType

          public void setValueType(Types.ValueType vt)

    - #### requiresRecompile

          public boolean requiresRecompile()

      Indicates if dynamic recompilation is required for this hop.
      Returns:
        true if dynamic recompilation required

    - #### setRequiresRecompile

          public void setRequiresRecompile()

      Marks the hop for dynamic recompilation.

    - #### refreshSizeInformation

          public abstract void refreshSizeInformation()

      Update the output size information for this hop.

    - #### computeSizeInformation

          public static long computeSizeInformation(Hop input)

    - #### refreshRowsParameterInformation

          public void refreshRowsParameterInformation(Hop input,
                                                      LocalVariableMap vars)

    - #### refreshRowsParameterInformation

          public void refreshRowsParameterInformation(Hop input,
                                                      LocalVariableMap vars,
                                                      HashMap<Long,Long> memo)

    - #### refreshColsParameterInformation

          public void refreshColsParameterInformation(Hop input,
                                                      LocalVariableMap vars)

    - #### refreshColsParameterInformation

          public void refreshColsParameterInformation(Hop input,
                                                      LocalVariableMap vars,
                                                      HashMap<Long,Long> memo)

    - #### computeSizeInformation

          public long computeSizeInformation(Hop input,
                                             LocalVariableMap vars)

    - #### computeSizeInformation

          public long computeSizeInformation(Hop input,
                                             LocalVariableMap vars,
                                             HashMap<Long,Long> memo)

    - #### computeBoundsInformation

          public double computeBoundsInformation(Hop input)

    - #### computeBoundsInformation

          public static double computeBoundsInformation(Hop input,
                                                        LocalVariableMap vars)

    - #### computeBoundsInformation

          public static double computeBoundsInformation(Hop input,
                                                        LocalVariableMap vars,
                                                        HashMap<Long,Double> memo)

    - #### clone

          public abstract Object clone()
                                throws CloneNotSupportedException

      Overrides:
        `clone` in class `Object`
      Throws:
        `CloneNotSupportedException`

    - #### compare

          public abstract boolean compare(Hop that)

    - #### setBeginLine

          public void setBeginLine(int passed)

      Specified by:
        `setBeginLine` in interface `ParseInfo`

    - #### setBeginColumn

          public void setBeginColumn(int passed)

      Specified by:
        `setBeginColumn` in interface `ParseInfo`

    - #### setEndLine

          public void setEndLine(int passed)

      Specified by:
        `setEndLine` in interface `ParseInfo`

    - #### setEndColumn

          public void setEndColumn(int passed)

      Specified by:
        `setEndColumn` in interface `ParseInfo`

    - #### setFilename

          public void setFilename(String passed)

      Specified by:
        `setFilename` in interface `ParseInfo`

    - #### setText

          public void setText(String text)

      Specified by:
        `setText` in interface `ParseInfo`

    - #### getBeginLine

          public int getBeginLine()

      Specified by:
        `getBeginLine` in interface `ParseInfo`

    - #### getBeginColumn

          public int getBeginColumn()

      Specified by:
        `getBeginColumn` in interface `ParseInfo`

    - #### getEndLine

          public int getEndLine()

      Specified by:
        `getEndLine` in interface `ParseInfo`

    - #### getEndColumn

          public int getEndColumn()

      Specified by:
        `getEndColumn` in interface `ParseInfo`

    - #### getFilename

          public String getFilename()

      Specified by:
        `getFilename` in interface `ParseInfo`

    - #### getText

          public String getText()

      Specified by:
        `getText` in interface `ParseInfo`

    - #### printErrorLocation

          public String printErrorLocation()

    - #### setParseInfo

          public void setParseInfo(ParseInfo parseInfo)

      Set parse information.
      Parameters:
        `parseInfo` - parse information, such as beginning line position, beginning column position, ending line position, ending column position, text, and filename

Copyright © 2021 The Apache Software Foundation. All rights reserved.