org.apache.sysds.lops

## Class Checkpoint

- java.lang.Object
- - org.apache.sysds.lops.Lop
  - - org.apache.sysds.lops.Checkpoint

-     public class Checkpoint
      extends Lop

  Lop for checkpoint operations. For example, on Spark, the semantic of a checkpoint is to persist an intermediate result into a specific storage level (e.g., mem_only). We use the name checkpoint in terms of cache/persist in Spark (not in terms of checkpoint in Spark streaming) in order to differentiate from CP caching. NOTE: since this class uses spark apis, it should only be instantiated if we are running in execution mode spark (whenever all spark libraries are available)

- - ### Nested Class Summary

    - ### Nested classes/interfaces inherited from class org.apache.sysds.lops.Lop

      `Lop.SimpleInstType, Lop.Type, Lop.VisitStatus`

  - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static boolean` | `CHECKPOINT_SPARSE_CSR`
    `static org.apache.spark.storage.StorageLevel` | `DEFAULT_STORAGE_LEVEL`
    `static String` | `OPCODE`
    `static org.apache.spark.storage.StorageLevel` | `SER_STORAGE_LEVEL`

    - ### Fields inherited from class org.apache.sysds.lops.Lop

      `_beginColumn, _beginLine, _endColumn, _endLine, _filename, CP_CHILD_THREAD, CP_ROOT_THREAD_ID, DATATYPE_PREFIX, FILE_SEPARATOR, FRAME_VAR_NAME_PREFIX, INSTRUCTION_DELIMITOR, LITERAL_PREFIX, MATRIX_VAR_NAME_PREFIX, NAME_VALUE_SEPARATOR, OPERAND_DELIMITOR, PROCESS_PREFIX, SAMPLE_FRACTION, SCALAR_VAR_NAME_PREFIX, UPDATE_INPLACE_PREFIX, VALUETYPE_PREFIX, VARIABLE_NAME_PLACEHOLDER`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `Checkpoint(Lop input, Types.DataType dt, Types.ValueType vt, String level)` TODO change string parameter storage.level to StorageLevel as soon as we can assume that Spark libraries are always available.

  - ### Method Summary

    All Methods Static Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `static String` | `getDefaultStorageLevelString()`
    `String` | `getInstructions(String input1, String output)` Method should be overridden if needed
    `static String` | `getSerializeStorageLevelString()`
    `org.apache.spark.storage.StorageLevel` | `getStorageLevel()`
    `static String` | `getStorageLevelString(org.apache.spark.storage.StorageLevel level)` This is a utility method because Sparks StorageLevel.toString() is incompatible with its own fromString() method.
    `void` | `setStorageLevel(org.apache.spark.storage.StorageLevel level)`
    `String` | `toString()` Method to have Lops print their state.

    - ### Methods inherited from class org.apache.sysds.lops.Lop

      `addInput, addOutput, addToDag, createReachable, getAggType, getBeginColumn, getBeginLine, getBroadcastInput, getDataType, getEndColumn, getEndLine, getExecType, getFilename, getID, getInputs, getInstructions, getInstructions, getInstructions, getInstructions, getInstructions, getInstructions, getInstructions, getInstructions, getInstructions, getInstructions, getInstructions, getLevel, getOutputParameters, getOutputs, getPrivacyConstraint, getProducesIntermediateOutput, getReachable, getSimpleInstructionType, getType, getValueType, getVisited, isAllOutputsCP, isDataExecLocation, isVariable, prepInputOperand, prepInputOperand, prepOperand, prepOperand, prepOutputOperand, prepOutputOperand, prepOutputOperand, prepScalarInputOperand, prepScalarInputOperand, prepScalarLabel, prepScalarOperand, printErrorLocation, removeConsumer, removeOutput, replaceInput, resetVisitStatus, setAllPositions, setBeginColumn, setBeginLine, setConsumerCount, setDataType, setEndColumn, setEndLine, setExecType, setFederatedOutput, setFilename, setPrivacyConstraint, setValueType, setVisited`

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, wait, wait, wait`

- - ### Field Detail

    - #### OPCODE

          public static final String OPCODE

      See Also:
        Constant Field Values

    - #### DEFAULT_STORAGE_LEVEL

          public static final org.apache.spark.storage.StorageLevel DEFAULT_STORAGE_LEVEL

    - #### SER_STORAGE_LEVEL

          public static final org.apache.spark.storage.StorageLevel SER_STORAGE_LEVEL

    - #### CHECKPOINT_SPARSE_CSR

          public static final boolean CHECKPOINT_SPARSE_CSR

      See Also:
        Constant Field Values

  - ### Constructor Detail

    - #### Checkpoint

          public Checkpoint(Lop input,
                            Types.DataType dt,
                            Types.ValueType vt,
                            String level)

      TODO change string parameter storage.level to StorageLevel as soon as we can assume that Spark libraries are always available.
      Parameters:
        `input` - low-level operator
        `dt` - data type
        `vt` - value type
        `level` - storage level

  - ### Method Detail

    - #### getStorageLevel

          public org.apache.spark.storage.StorageLevel getStorageLevel()

    - #### setStorageLevel

          public void setStorageLevel(org.apache.spark.storage.StorageLevel level)

    - #### toString

          public String toString()

      Description copied from class: `Lop`
      Method to have Lops print their state. This is for debugging purposes.
      Specified by:
        `toString` in class `Lop`

    - #### getInstructions

          public String getInstructions(String input1,
                                        String output)

      Description copied from class: `Lop`
      Method should be overridden if needed
      Overrides:
        `getInstructions` in class `Lop`
      Parameters:
        `input1` - input 1
        `output` - output
      Returns:
        instructions as string

    - #### getStorageLevelString

          public static String getStorageLevelString(org.apache.spark.storage.StorageLevel level)

      This is a utility method because Sparks StorageLevel.toString() is incompatible with its own fromString() method.
      Parameters:
        `level` - RDD storage level
      Returns:
        storage level as a string

    - #### getDefaultStorageLevelString

          public static String getDefaultStorageLevelString()

    - #### getSerializeStorageLevelString

          public static String getSerializeStorageLevelString()

Copyright © 2021 The Apache Software Foundation. All rights reserved.