Package org.apache.sysds.hops

## Class DataOp

- java.lang.Object
- - org.apache.sysds.hops.Hop
  - - org.apache.sysds.hops.DataOp

- All Implemented Interfaces:
    `ParseInfo`

      public class DataOp
      extends Hop

  A DataOp can be either a persistent read/write or transient read/write - writes will always have at least one input, but all types can have parameters (e.g., for csv literals of delimiter, header, etc).

- - ### Field Summary

    - ### Fields inherited from class org.apache.sysds.hops.Hop

      `_beginColumn, _beginLine, _endColumn, _endLine, _filename, _text, CPThreshold`

  - ### Constructor Summary

    Constructors
    Constructor | Description
    --- | ---
    `DataOp​(String l, Types.DataType dt, Types.ValueType vt, Types.OpOpData dop, String fname, long dim1, long dim2, long nnz, int blen)` | READ operation for Matrix w/ dim1, dim2.
    `DataOp​(String l, Types.DataType dt, Types.ValueType vt, Types.OpOpData dop, String fname, long dim1, long dim2, long nnz, MatrixObject.UpdateType update, int blen)` |
    `DataOp​(String l, Types.DataType dt, Types.ValueType vt, Types.OpOpData dop, HashMap<String,​Hop> params)` | READ operation for Matrix / SQL operation for tensor This constructor supports expressions in parameters
    `DataOp​(String l, Types.DataType dt, Types.ValueType vt, Types.OpOpData dop, Hop in, HashMap<String,​Hop> inputParameters)` | WRITE operation for Matrix This constructor supports expression in parameters
    `DataOp​(String l, Types.DataType dt, Types.ValueType vt, Hop in, Types.OpOpData dop, String fname)` |

  - ### Method Summary

    All Methods Instance Methods Concrete Methods
    Modifier and Type | Method | Description
    --- | --- | ---
    `boolean` | `allowsAllExecTypes()` |
    `void` | `checkArity()` | Check for N (READ) or N+1 (WRITE) inputs.
    `Object` | `clone()` |
    `boolean` | `compare​(Hop that)` |
    `Lop` | `constructLops()` |
    `void` | `disableRecompileRead()` | Explicitly disables recompilation of transient reads, this additional information is required because requiresRecompile is set in a top-down manner, hence any value set from a consuming operating would be overwritten by opFindExecType.
    `Types.FileFormat` | `getFileFormat()` |
    `String` | `getFileName()` |
    `long` | `getInputBlocksize()` |
    `Types.OpOpData` | `getOp()` |
    `String` | `getOpString()` |
    `int` | `getParameterIndex​(String name)` |
    `boolean` | `hasOnlyRDD()` |
    `boolean` | `isFederatedDataOp()` | Checks if the hop is a DataOp with federated data.
    `boolean` | `isGPUEnabled()` | In memory-based optimizer mode (see OptimizerUtils.isMemoryBasedOptLevel()), the exectype is determined by checking this method as well as memory budget of this Hop.
    `boolean` | `isPersistentReadWrite()` |
    `boolean` | `isRead()` |
    `boolean` | `isWrite()` |
    `void` | `refreshSizeInformation()` | Update the output size information for this hop.
    `void` | `removeInput​(String inputName)` | Remove an input from the list of inputs and from the parameter index map.
    `void` | `setDataOpType​(Types.OpOpData type)` |
    `void` | `setFileFormat​(Types.FileFormat ft)` |
    `void` | `setFileName​(String fn)` |
    `void` | `setInputBlocksize​(long blen)` |
    `void` | `setOnlyRDD​(boolean flag)` |
    `void` | `setOutputParams​(long dim1, long dim2, long nnz, MatrixObject.UpdateType update, int blen)` |

    - ### Methods inherited from class org.apache.sysds.hops.Hop

      `activatePrefetch, addAllInputs, addInput, checkAndSetForcedPlatform, checkAndSetInvalidCPDimsAndSize, clearMemEstimate, colsKnown, compressedSize, computeBoundsInformation, computeBoundsInformation, computeBoundsInformation, computeMemEstimate, computeSizeInformation, computeSizeInformation, computeSizeInformation, constructAndSetLopsDataFlowProperties, createOffsetLop, deactivatePrefetch, dimsKnown, dimsKnown, dimsKnownAny, federatedCostInitialized, getBeginColumn, getBeginLine, getBlocksize, getCompressedSize, getDataCharacteristics, getDataType, getDim, getDim1, getDim2, getEndColumn, getEndLine, getExecType, getFederatedCost, getFederatedOutput, getFilename, getForcedExecType, getHopID, getInput, getInput, getInputMemEstimate, getInputMemEstimate, getInputOutputSize, getIntermediateMemEstimate, getLength, getLops, getMemEstimate, getName, getNnz, getOutputMemEstimate, getOutputMemEstimate, getParent, getPrivacy, getRepetitions, getSparsity, getSpBroadcastSize, getText, getUpdateType, getValueType, hasCompressedInput, hasFederatedOutput, hasLocalOutput, hasMatrixInputWithDifferentBlocksizes, hasValidCPDimsAndSize, isCompressedOutput, isFederated, isMatrix, isMemEstimated, isOutputEmptyBlocks, isRequiredDecompression, isScalar, isTransposeSafe, isVisited, prefetchActivated, printErrorLocation, refreshColsParameterInformation, refreshColsParameterInformation, refreshMemEstimates, refreshRowsParameterInformation, refreshRowsParameterInformation, requiresCheckpoint, requiresCompression, requiresLineageCaching, requiresReblock, requiresRecompile, resetExecType, resetFederatedCost, resetRecompilationFlag, resetRecompilationFlag, resetVisitStatus, resetVisitStatus, resetVisitStatus, resetVisitStatusForced, rowsKnown, setBeginColumn, setBeginLine, setBlocksize, setCompressedOutput, setCompressedSize, setDataType, setDim, setDim1, setDim2, setEndColumn, setEndLine, setExecType, setFederatedCost, setFederatedOutput, setFilename, setForcedExecType, setLops, setMemEstimate, setName, setNnz, setOutputEmptyBlocks, setParseInfo, setPrivacy, setRequiresCheckpoint, setRequiresCompression, setRequiresCompression, setRequiresDeCompression, setRequiresLineageCaching, setRequiresReblock, setRequiresRecompile, setText, setUpdateType, setValueType, setVisited, setVisited, someInputFederated, toString, updateLopFedOut, updateLopFedOut, updateRepetitionEstimates`

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, wait, wait, wait`

- - ### Constructor Detail

    - #### DataOp

          public DataOp​(String l,
                        Types.DataType dt,
                        Types.ValueType vt,
                        Types.OpOpData dop,
                        String fname,
                        long dim1,
                        long dim2,
                        long nnz,
                        int blen)

      READ operation for Matrix w/ dim1, dim2. This constructor does not support any expression in parameters
      Parameters:
        `l` - ?
        `dt` - data type
        `vt` - value type
        `dop` - data operator type
        `fname` - file name
        `dim1` - dimension 1
        `dim2` - dimension 2
        `nnz` - number of non-zeros
        `blen` - rows/cols per block

    - #### DataOp

          public DataOp​(String l,
                        Types.DataType dt,
                        Types.ValueType vt,
                        Types.OpOpData dop,
                        String fname,
                        long dim1,
                        long dim2,
                        long nnz,
                        MatrixObject.UpdateType update,
                        int blen)

    - #### DataOp

          public DataOp​(String l,
                        Types.DataType dt,
                        Types.ValueType vt,
                        Types.OpOpData dop,
                        HashMap<String,​Hop> params)

      READ operation for Matrix / SQL operation for tensor This constructor supports expressions in parameters
      Parameters:
        `l` - ?
        `dt` - data type
        `vt` - value type
        `dop` - data operator type
        `params` - input parameters

    - #### DataOp

          public DataOp​(String l,
                        Types.DataType dt,
                        Types.ValueType vt,
                        Hop in,
                        Types.OpOpData dop,
                        String fname)

    - #### DataOp

          public DataOp​(String l,
                        Types.DataType dt,
                        Types.ValueType vt,
                        Types.OpOpData dop,
                        Hop in,
                        HashMap<String,​Hop> inputParameters)

      WRITE operation for Matrix This constructor supports expression in parameters
      Parameters:
        `l` - ?
        `dt` - data type
        `vt` - value type
        `dop` - data operator type
        `in` - high-level operator
        `inputParameters` - input parameters

  - ### Method Detail

    - #### checkArity

          public void checkArity()

      Check for N (READ) or N+1 (WRITE) inputs.
      Specified by:
        `checkArity` in class `Hop`

    - #### getOp

          public Types.OpOpData getOp()

    - #### setDataOpType

          public void setDataOpType​(Types.OpOpData type)

    - #### setOutputParams

          public void setOutputParams​(long dim1,
                                      long dim2,
                                      long nnz,
                                      MatrixObject.UpdateType update,
                                      int blen)

    - #### setFileName

          public void setFileName​(String fn)

    - #### getFileName

          public String getFileName()

    - #### getParameterIndex

          public int getParameterIndex​(String name)

    - #### setOnlyRDD

          public void setOnlyRDD​(boolean flag)

    - #### hasOnlyRDD

          public boolean hasOnlyRDD()

    - #### isGPUEnabled

          public boolean isGPUEnabled()

      Description copied from class: `Hop`
      In memory-based optimizer mode (see OptimizerUtils.isMemoryBasedOptLevel()), the exectype is determined by checking this method as well as memory budget of this Hop. Please see findExecTypeByMemEstimate for more detail. This method is necessary because not all operator are supported efficiently on GPU (for example: operations on frames and scalar as well as operations such as table).
      Specified by:
        `isGPUEnabled` in class `Hop`
      Returns:
        true if the Hop is eligible for GPU Exectype.

    - #### constructLops

          public Lop constructLops()

      Specified by:
        `constructLops` in class `Hop`

    - #### setFileFormat

          public void setFileFormat​(Types.FileFormat ft)

    - #### getFileFormat

          public Types.FileFormat getFileFormat()

    - #### setInputBlocksize

          public void setInputBlocksize​(long blen)

    - #### getInputBlocksize

          public long getInputBlocksize()

    - #### isRead

          public boolean isRead()

    - #### isWrite

          public boolean isWrite()

    - #### isPersistentReadWrite

          public boolean isPersistentReadWrite()

    - #### isFederatedDataOp

          public boolean isFederatedDataOp()

      Description copied from class: `Hop`
      Checks if the hop is a DataOp with federated data.
      Overrides:
        `isFederatedDataOp` in class `Hop`
      Returns:
        true if hop is a federated DataOp

    - #### getOpString

          public String getOpString()

      Specified by:
        `getOpString` in class `Hop`

    - #### allowsAllExecTypes

          public boolean allowsAllExecTypes()

      Specified by:
        `allowsAllExecTypes` in class `Hop`

    - #### refreshSizeInformation

          public void refreshSizeInformation()

      Description copied from class: `Hop`
      Update the output size information for this hop.
      Specified by:
        `refreshSizeInformation` in class `Hop`

    - #### disableRecompileRead

          public void disableRecompileRead()

      Explicitly disables recompilation of transient reads, this additional information is required because requiresRecompile is set in a top-down manner, hence any value set from a consuming operating would be overwritten by opFindExecType.

    - #### clone

          public Object clone()
                       throws CloneNotSupportedException

      Specified by:
        `clone` in class `Hop`
      Throws:
        `CloneNotSupportedException`

    - #### compare

          public boolean compare​(Hop that)

      Specified by:
        `compare` in class `Hop`

    - #### removeInput

          public void removeInput​(String inputName)

      Remove an input from the list of inputs and from the parameter index map. Parameter index map values higher than the index of the removed input will be decremented by one.
      Parameters:
        `inputName` - The name of the input to remove