org.apache.sysds.runtime.controlprogram.caching

## Class MatrixObject

- java.lang.Object
- - org.apache.sysds.runtime.instructions.cp.Data
  - - org.apache.sysds.runtime.controlprogram.caching.CacheableData<MatrixBlock>
    - - org.apache.sysds.runtime.controlprogram.caching.MatrixObject

- All Implemented Interfaces:
    Serializable

      public class MatrixObject
      extends CacheableData<MatrixBlock>

  Represents a matrix in control program. This class contains method to read matrices from HDFS and convert them to a specific format/representation. It is also able to write several formats/representation of matrices to HDFS. IMPORTANT: Preserve one-to-one correspondence between `MatrixObject` and `MatrixBlock` objects, for cache purposes. Do not change a `MatrixBlock` object without informing its `MatrixObject` object.
  See Also:
    Serialized Form

- - ### Nested Class Summary

    Nested Classes
    Modifier and Type | Class and Description
    --- | ---
    `static class` | `MatrixObject.UpdateType`

    - ### Nested classes/interfaces inherited from class org.apache.sysds.runtime.controlprogram.caching.CacheableData

      `CacheableData.CacheStatus`

  - ### Field Summary

    - ### Fields inherited from class org.apache.sysds.runtime.controlprogram.caching.CacheableData

      `cacheEvictionLocalFilePath, cacheEvictionLocalFilePrefix, CACHING_ASYNC_FILECLEANUP, CACHING_ASYNC_SERIALIZE, CACHING_BUFFER_PAGECACHE, CACHING_BUFFER_POLICY, CACHING_BUFFER_SIZE, CACHING_COUNTER_GROUP_NAME, CACHING_EVICTION_FILEEXTENSION, CACHING_THRESHOLD, CACHING_WRITE_CACHE_ON_READ`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `MatrixObject(MatrixObject mo)` Copy constructor that copies meta data but NO data.
    `MatrixObject(Types.ValueType vt, String file)` Constructor that takes the value type and the HDFS filename.
    `MatrixObject(Types.ValueType vt, String file, MetaData mtd)` Constructor that takes the value type, HDFS filename and associated metadata.
    `MatrixObject(Types.ValueType vt, String file, MetaData mtd, MatrixBlock data)` Constructor that takes the value type, HDFS filename and associated metadata and a MatrixBlock used for creation after serialization

  - ### Method Summary

    All Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `long` | `getBlocksize()`
    `long` | `getNnz()`
    `String` | `getPartitionFileName(IndexRange pred, int blen)`
    `ParForProgramBlock.PDataPartitionFormat` | `getPartitionFormat()`
    `int` | `getPartitionSize()`
    `double` | `getSparsity()`
    `MatrixObject.UpdateType` | `getUpdateType()`
    `boolean` | `isDiag()`
    `boolean` | `isMarked()`
    `boolean` | `isPartitioned()`
    `MatrixBlock` | `readMatrixPartition(IndexRange pred)` NOTE: for reading matrix partitions, we could cache (in its real sense) the read block with soft references (no need for eviction, as partitioning only applied for read-only matrices).
    `void` | `refreshMetaData()` Make the matrix metadata consistent with the in-memory matrix data
    `void` | `setDiag(boolean diag)`
    `void` | `setInMemoryPartition(MatrixBlock block)`
    `void` | `setMarkForLinCache(boolean mark)`
    `void` | `setPartitioned(ParForProgramBlock.PDataPartitionFormat format, int n)`
    `void` | `setUpdateType(MatrixObject.UpdateType flag)`
    `void` | `unsetPartitioned()`
    `void` | `updateDataCharacteristics(DataCharacteristics dc)`

    - ### Methods inherited from class org.apache.sysds.runtime.controlprogram.caching.CacheableData

      `acquireModify, acquireRead, acquireReadAndRelease, addBroadcastSize, cleanupCacheDir, cleanupCacheDir, clearData, clearData, disableCaching, enableCaching, enableCleanup, exportData, exportData, exportData, exportData, exportData, exportData, freeEvictedBlob, getBroadcastHandle, getBroadcastSize, getCacheLineage, getDataCharacteristics, getDataSize, getDebugName, getFedMapping, getFileFormatProperties, getFileName, getGPUObject, getMetaData, getNumColumns, getNumRows, getRDDHandle, getStatus, getUniqueID, hasValidLineage, initCaching, initCaching, isBelowCachingThreshold, isCached, isCachingActive, isCleanupEnabled, isDirty, isFederated, isFederated, isHDFSFileExists, moveData, release, removeGPUObject, removeMetaData, setBroadcastHandle, setCacheLineage, setDirty, setEmptyStatus, setFedMapping, setFileFormatProperties, setFileName, setGPUObject, setHDFSFileExists, setMetaData, setRDDHandle, toString`

    - ### Methods inherited from class org.apache.sysds.runtime.instructions.cp.Data

      `getDataType, getPrivacyConstraint, getValueType, setPrivacyConstraints`

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, wait, wait, wait`

- - ### Constructor Detail

    - #### MatrixObject

          public MatrixObject(Types.ValueType vt,
                              String file)

      Constructor that takes the value type and the HDFS filename.
      Parameters:
        `vt` - value type
        `file` - file name

    - #### MatrixObject

          public MatrixObject(Types.ValueType vt,
                              String file,
                              MetaData mtd)

      Constructor that takes the value type, HDFS filename and associated metadata.
      Parameters:
        `vt` - value type
        `file` - file name
        `mtd` - metadata

    - #### MatrixObject

          public MatrixObject(Types.ValueType vt,
                              String file,
                              MetaData mtd,
                              MatrixBlock data)

      Constructor that takes the value type, HDFS filename and associated metadata and a MatrixBlock used for creation after serialization
      Parameters:
        `vt` - value type
        `file` - file name
        `mtd` - metadata
        `data` - matrix block data

    - #### MatrixObject

          public MatrixObject(MatrixObject mo)

      Copy constructor that copies meta data but NO data.
      Parameters:
        `mo` - matrix object

  - ### Method Detail

    - #### setUpdateType

          public void setUpdateType(MatrixObject.UpdateType flag)

    - #### getUpdateType

          public MatrixObject.UpdateType getUpdateType()

    - #### isDiag

          public boolean isDiag()

    - #### setDiag

          public void setDiag(boolean diag)

    - #### setMarkForLinCache

          public void setMarkForLinCache(boolean mark)

    - #### isMarked

          public boolean isMarked()

    - #### updateDataCharacteristics

          public void updateDataCharacteristics(DataCharacteristics dc)

      Overrides:
        `updateDataCharacteristics` in class `Data`

    - #### refreshMetaData

          public void refreshMetaData()

      Make the matrix metadata consistent with the in-memory matrix data
      Specified by:
        `refreshMetaData` in class `CacheableData<MatrixBlock>`

    - #### getBlocksize

          public long getBlocksize()

      Overrides:
        `getBlocksize` in class `CacheableData<MatrixBlock>`

    - #### getNnz

          public long getNnz()

    - #### getSparsity

          public double getSparsity()

    - #### setPartitioned

          public void setPartitioned(ParForProgramBlock.PDataPartitionFormat format,
                                     int n)

    - #### unsetPartitioned

          public void unsetPartitioned()

    - #### isPartitioned

          public boolean isPartitioned()

    - #### getPartitionFormat

          public ParForProgramBlock.PDataPartitionFormat getPartitionFormat()

    - #### getPartitionSize

          public int getPartitionSize()

    - #### setInMemoryPartition

          public void setInMemoryPartition(MatrixBlock block)

    - #### readMatrixPartition

          public MatrixBlock readMatrixPartition(IndexRange pred)

      NOTE: for reading matrix partitions, we could cache (in its real sense) the read block with soft references (no need for eviction, as partitioning only applied for read-only matrices). However, since we currently only support row- and column-wise partitioning caching is not applied yet. This could be changed once we also support column-block-wise and row-block-wise. Furthermore, as we reject to partition vectors and support only full row or column indexing, no metadata (apart from the partition flag) is required.
      Parameters:
        `pred` - index range
      Returns:
        matrix block

    - #### getPartitionFileName

          public String getPartitionFileName(IndexRange pred,
                                             int blen)

Copyright © 2021 The Apache Software Foundation. All rights reserved.