Package org.apache.sysds.runtime.controlprogram.caching

## Class MatrixObject

- java.lang.Object
- - org.apache.sysds.runtime.instructions.cp.Data
  - - org.apache.sysds.runtime.controlprogram.caching.CacheableData<MatrixBlock>
    - - org.apache.sysds.runtime.controlprogram.caching.MatrixObject

- All Implemented Interfaces:
    `Serializable`

      public class MatrixObject
      extends CacheableData<MatrixBlock>

  Represents a matrix in control program. This class contains method to read matrices from HDFS and convert them to a specific format/representation. It is also able to write several formats/representation of matrices to HDFS. IMPORTANT: Preserve one-to-one correspondence between `MatrixObject` and `MatrixBlock` objects, for cache purposes. Do not change a `MatrixBlock` object without informing its `MatrixObject` object.
  See Also:
    Serialized Form

- - ### Nested Class Summary

    Nested Classes
    Modifier and Type | Class | Description
    --- | --- | ---
    `static class` | `MatrixObject.UpdateType` |

    - ### Nested classes/interfaces inherited from class org.apache.sysds.runtime.controlprogram.caching.CacheableData

      `CacheableData.CacheStatus`

  - ### Field Summary

    - ### Fields inherited from class org.apache.sysds.runtime.controlprogram.caching.CacheableData

      `cacheEvictionLocalFilePath, cacheEvictionLocalFilePrefix, CACHING_ASYNC_FILECLEANUP, CACHING_ASYNC_SERIALIZE, CACHING_BUFFER_PAGECACHE, CACHING_BUFFER_POLICY, CACHING_COUNTER_GROUP_NAME, CACHING_EVICTION_FILEEXTENSION, CACHING_THRESHOLD, CACHING_WRITE_CACHE_ON_READ`

  - ### Constructor Summary

    Constructors
    Constructor | Description
    --- | ---
    `MatrixObject​(Types.ValueType vt, String file)` | Constructor that takes the value type and the HDFS filename.
    `MatrixObject​(Types.ValueType vt, String file, MetaData mtd)` | Constructor that takes the value type, HDFS filename and associated metadata.
    `MatrixObject​(Types.ValueType vt, String file, MetaData mtd, MatrixBlock data)` | Constructor that takes the value type, HDFS filename and associated metadata and a MatrixBlock used for creation after serialization
    `MatrixObject​(MatrixObject mo)` | Copy constructor that copies meta data but NO data.

  - ### Method Summary

    All Methods Instance Methods Concrete Methods
    Modifier and Type | Method | Description
    --- | --- | ---
    `long` | `getBlocksize()` |
    `long` | `getNnz()` |
    `String` | `getPartitionFileName​(IndexRange pred, int blen)` |
    `ParForProgramBlock.PDataPartitionFormat` | `getPartitionFormat()` |
    `int` | `getPartitionSize()` |
    `double` | `getSparsity()` |
    `MatrixObject.UpdateType` | `getUpdateType()` |
    `boolean` | `isDiag()` |
    `boolean` | `isMarked()` |
    `boolean` | `isPartitioned()` |
    `MatrixBlock` | `readMatrixPartition​(IndexRange pred)` | NOTE: for reading matrix partitions, we could cache (in its real sense) the read block with soft references (no need for eviction, as partitioning only applied for read-only matrices).
    `void` | `refreshMetaData()` | Make the matrix metadata consistent with the in-memory matrix data
    `void` | `setDiag​(boolean diag)` |
    `void` | `setInMemoryPartition​(MatrixBlock block)` |
    `void` | `setMarkForLinCache​(boolean mark)` |
    `void` | `setPartitioned​(ParForProgramBlock.PDataPartitionFormat format, int n)` |
    `void` | `setUpdateType​(MatrixObject.UpdateType flag)` |
    `String` | `toString()` |
    `void` | `unsetPartitioned()` |
    `void` | `updateDataCharacteristics​(DataCharacteristics dc)` |

    - ### Methods inherited from class org.apache.sysds.runtime.controlprogram.caching.CacheableData

      `acquireModify, acquireRead, acquireReadAndRelease, addBroadcastSize, cleanupCacheDir, cleanupCacheDir, clearData, clearData, disableCaching, enableCaching, enableCleanup, exportData, exportData, exportData, exportData, exportData, freeEvictedBlob, getBroadcastHandle, getBroadcastSize, getCacheLineage, getCompressedSize, getDataCharacteristics, getDataSize, getDebugName, getDim, getFedMapping, getFileFormatProperties, getFileName, getGPUObject, getMetaData, getNumColumns, getNumRows, getRDDHandle, getStatus, getUniqueID, hasValidLineage, initCaching, initCaching, isBelowCachingThreshold, isCached, isCachingActive, isCleanupEnabled, isCompressed, isDirty, isFederated, isFederated, isFederatedExcept, isHDFSFileExists, isPendingRDDOps, moveData, release, removeGPUObject, removeMetaData, setBroadcastHandle, setCacheLineage, setCompressedSize, setDirty, setEmptyStatus, setFedMapping, setFileFormatProperties, setFileName, setGPUObject, setHDFSFileExists, setMetaData, setRDDHandle`

    - ### Methods inherited from class org.apache.sysds.runtime.instructions.cp.Data

      `getDataType, getPrivacyConstraint, getValueType, setPrivacyConstraints`

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, wait, wait, wait`

- - ### Constructor Detail

    - #### MatrixObject

          public MatrixObject​(Types.ValueType vt,
                              String file)

      Constructor that takes the value type and the HDFS filename.
      Parameters:
        `vt` - value type
        `file` - file name

    - #### MatrixObject

          public MatrixObject​(Types.ValueType vt,
                              String file,
                              MetaData mtd)

      Constructor that takes the value type, HDFS filename and associated metadata.
      Parameters:
        `vt` - value type
        `file` - file name
        `mtd` - metadata

    - #### MatrixObject

          public MatrixObject​(Types.ValueType vt,
                              String file,
                              MetaData mtd,
                              MatrixBlock data)

      Constructor that takes the value type, HDFS filename and associated metadata and a MatrixBlock used for creation after serialization
      Parameters:
        `vt` - value type
        `file` - file name
        `mtd` - metadata
        `data` - matrix block data

    - #### MatrixObject

          public MatrixObject​(MatrixObject mo)

      Copy constructor that copies meta data but NO data.
      Parameters:
        `mo` - matrix object

  - ### Method Detail

    - #### setUpdateType

          public void setUpdateType​(MatrixObject.UpdateType flag)

    - #### getUpdateType

          public MatrixObject.UpdateType getUpdateType()

    - #### isDiag

          public boolean isDiag()

    - #### setDiag

          public void setDiag​(boolean diag)

    - #### setMarkForLinCache

          public void setMarkForLinCache​(boolean mark)

    - #### isMarked

          public boolean isMarked()

    - #### updateDataCharacteristics

          public void updateDataCharacteristics​(DataCharacteristics dc)

      Overrides:
        `updateDataCharacteristics` in class `Data`

    - #### refreshMetaData

          public void refreshMetaData()

      Make the matrix metadata consistent with the in-memory matrix data
      Specified by:
        `refreshMetaData` in class `CacheableData<MatrixBlock>`

    - #### getBlocksize

          public long getBlocksize()

      Overrides:
        `getBlocksize` in class `CacheableData<MatrixBlock>`

    - #### getNnz

          public long getNnz()

    - #### getSparsity

          public double getSparsity()

    - #### setPartitioned

          public void setPartitioned​(ParForProgramBlock.PDataPartitionFormat format,
                                     int n)

    - #### unsetPartitioned

          public void unsetPartitioned()

    - #### isPartitioned

          public boolean isPartitioned()

    - #### getPartitionFormat

          public ParForProgramBlock.PDataPartitionFormat getPartitionFormat()

    - #### getPartitionSize

          public int getPartitionSize()

    - #### setInMemoryPartition

          public void setInMemoryPartition​(MatrixBlock block)

    - #### readMatrixPartition

          public MatrixBlock readMatrixPartition​(IndexRange pred)

      NOTE: for reading matrix partitions, we could cache (in its real sense) the read block with soft references (no need for eviction, as partitioning only applied for read-only matrices). However, since we currently only support row- and column-wise partitioning caching is not applied yet. This could be changed once we also support column-block-wise and row-block-wise. Furthermore, as we reject to partition vectors and support only full row or column indexing, no metadata (apart from the partition flag) is required.
      Parameters:
        `pred` - index range
      Returns:
        matrix block

    - #### getPartitionFileName

          public String getPartitionFileName​(IndexRange pred,
                                             int blen)

    - #### toString

          public String toString()

      Overrides:
        `toString` in class `CacheableData<MatrixBlock>`