org.apache.sysds.runtime.compress

## Class CompressedMatrixBlock

- java.lang.Object
- - org.apache.sysds.runtime.matrix.data.MatrixValue
  - - org.apache.sysds.runtime.matrix.data.MatrixBlock
    - - org.apache.sysds.runtime.compress.AbstractCompressedMatrixBlock
      - - org.apache.sysds.runtime.compress.CompressedMatrixBlock

- All Implemented Interfaces:
    Externalizable, Serializable, Comparable, org.apache.hadoop.io.Writable, org.apache.hadoop.io.WritableComparable, CacheBlock

      public class CompressedMatrixBlock
      extends AbstractCompressedMatrixBlock

  See Also:
    Serialized Form

- - ### Nested Class Summary

    - ### Nested classes/interfaces inherited from class org.apache.sysds.runtime.matrix.data.MatrixBlock

      `MatrixBlock.SparsityEstimate`

    - ### Nested classes/interfaces inherited from class org.apache.sysds.runtime.matrix.data.MatrixValue

      `MatrixValue.CellIndex`

  - ### Field Summary

    - ### Fields inherited from class org.apache.sysds.runtime.matrix.data.MatrixBlock

      `CONVERT_MCSR_TO_CSR_ON_DEEP_SERIALIZE, DEFAULT_INPLACE_SPARSEBLOCK, DEFAULT_SPARSEBLOCK, HEADER_SIZE, MAX_SHALLOW_SERIALIZE_OVERHEAD, SPARSITY_TURN_POINT, ULTRA_SPARSE_BLOCK_NNZ, ULTRA_SPARSITY_TURN_POINT, ULTRA_SPARSITY_TURN_POINT2`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `CompressedMatrixBlock()` Constructor for building an empty Compressed Matrix block object.

  - ### Method Summary

    All Methods Static Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `MatrixBlock` | `aggregateBinaryOperations(MatrixBlock m1, MatrixBlock m2, MatrixBlock ret, AggregateBinaryOperator op)`
    `MatrixBlock` | `aggregateUnaryOperations(AggregateUnaryOperator op, MatrixValue result, int blen, MatrixIndexes indexesIn)`
    `MatrixBlock` | `aggregateUnaryOperations(AggregateUnaryOperator op, MatrixValue result, int blen, MatrixIndexes indexesIn, boolean inCP)`
    `void` | `allocateColGroupList(List<ColGroup> colGroups)`
    `MatrixBlock` | `append(MatrixBlock that, MatrixBlock ret)`
    `static long` | `baseSizeInMemory()`
    `MatrixBlock` | `chainMatrixMultOperations(MatrixBlock v, MatrixBlock w, MatrixBlock out, MapMultChain.ChainType ctype)`
    `MatrixBlock` | `chainMatrixMultOperations(MatrixBlock v, MatrixBlock w, MatrixBlock out, MapMultChain.ChainType ctype, int k)`
    `int[]` | `countNonZerosPerRow(int rl, int ru)`
    `MatrixBlock` | `decompress()` Decompress block.
    `MatrixBlock` | `decompress(int k)` Decompress block.
    `long` | `estimateCompressedSizeInMemory()` Obtain an upper bound on the memory used to store the compressed block.
    `List<ColGroup>` | `getColGroups()`
    `Iterator<double[]>` | `getDenseRowIterator(int rl, int ru)`
    `long` | `getExactSizeOnDisk()` NOTE: The used estimates must be kept consistent with the respective write functions.
    `Iterator<IJV>` | `getIterator(int rl, int ru, boolean inclZeros)`
    `Iterator<IJV>` | `getIterator(int rl, int ru, int cgl, int cgu, boolean inclZeros)`
    `Iterator<SparseRow>` | `getSparseRowIterator(int rl, int ru)`
    `boolean` | `hasUncompressedColGroup()`
    `boolean` | `isSingleUncompressedGroup()`
    `double` | `quickGetValue(int r, int c)`
    `void` | `readExternal(ObjectInput is)` Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd de-serialization.
    `void` | `readFields(DataInput in)`
    `MatrixBlock` | `scalarOperations(ScalarOperator sop, MatrixValue result)`
    `String` | `toString()`
    `MatrixBlock` | `transposeSelfMatrixMultOperations(MatrixBlock out, MMTSJ.MMTSJType tstype)`
    `MatrixBlock` | `transposeSelfMatrixMultOperations(MatrixBlock out, MMTSJ.MMTSJType tstype, int k)`
    `void` | `write(DataOutput out)`
    `void` | `writeExternal(ObjectOutput os)` Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd serialization.

    - ### Methods inherited from class org.apache.sysds.runtime.compress.AbstractCompressedMatrixBlock

      `aggregateBinaryOperations, aggregateTernaryOperations, append, append, binaryOperations, binaryOperationsInPlace, cmOperations, cmOperations, covOperations, covOperations, ctableOperations, ctableOperations, ctableOperations, ctableOperations, ctableOperations, ctableOperations, ctableSeqOperations, estimateOriginalSizeInMemory, groupedAggOperations, groupedAggOperations, incrementalAggregate, incrementalAggregate, isEmptyBlock, isShallowSerialize, isShallowSerialize, leftIndexingOperations, leftIndexingOperations, permutationMatrixMultOperations, permutationMatrixMultOperations, quaternaryOperations, quaternaryOperations, randOperationsInPlace, randOperationsInPlace, removeEmptyOperations, removeEmptyOperations, reorgOperations, replaceOperations, rexpandOperations, seqOperationsInPlace, slice, slice, sortOperations, ternaryOperations, toShallowSerializeBlock, uaggouterchainOperations, unaryOperations, zeroOutOperations`

    - ### Methods inherited from class org.apache.sysds.runtime.matrix.data.MatrixBlock

      `allocateAndResetSparseBlock, allocateBlock, allocateBlockAsync, allocateDenseBlock, allocateDenseBlock, allocateDenseBlockUnsafe, allocateSparseRowsBlock, allocateSparseRowsBlock, append, appendRow, appendRow, appendRowToSparse, appendToSparse, appendToSparse, appendValue, checkNaN, checkNonZeros, checkSparseRows, checkSparseRows, cleanupBlock, compactEmptyBlock, compareTo, computeIQMCorrection, containsValue, copy, copy, copy, copyShallow, dropLastRowsOrColumns, equals, estimateSizeDenseInMemory, estimateSizeInMemory, estimateSizeInMemory, estimateSizeOnDisk, estimateSizeOnDisk, estimateSizeSparseInMemory, estimateSizeSparseInMemory, estimateSparsityOnAggBinary, evalSparseFormatInMemory, evalSparseFormatInMemory, evalSparseFormatInMemory, evalSparseFormatOnDisk, evalSparseFormatOnDisk, examSparsity, examSparsity, extractTriangular, getDataCharacteristics, getDenseBlock, getDenseBlockValues, getExactSerializedSize, getInMemorySize, getLength, getNonZeros, getNumColumns, getNumRows, getSparseBlock, getSparseBlockIterator, getSparseBlockIterator, getSparsity, getValue, getValueDenseUnsafe, hashCode, init, init, interQuartileMean, isAllocated, isEmpty, isEmptyBlock, isInSparseFormat, isSparsePermutationMatrix, isThreadSafe, isThreadSafe, isUltraSparse, isUltraSparse, isVector, leftIndexingOperations, max, mean, median, merge, merge, merge, merge, min, minNonZero, naryOperations, pickValue, pickValue, pickValues, prod, quickSetValue, randOperations, randOperations, randOperations, randOperations, recomputeNonZeros, recomputeNonZeros, recomputeNonZeros, reset, reset, reset, reset, reset, reset, sampleOperations, seqOperations, setNonZeros, setNumColumns, setNumRows, setSparseBlock, setValue, slice, slice, slice, sortSparseRows, sortSparseRows, sparseToDense, sum, sumSq, sumWeightForQuantile`

    - ### Methods inherited from class java.lang.Object

      `getClass, notify, notifyAll, wait, wait, wait`

- - ### Constructor Detail

    - #### CompressedMatrixBlock

          public CompressedMatrixBlock()

      Constructor for building an empty Compressed Matrix block object. OBS! Only to be used for serialization.

  - ### Method Detail

    - #### isSingleUncompressedGroup

          public boolean isSingleUncompressedGroup()

    - #### allocateColGroupList

          public void allocateColGroupList(List<ColGroup> colGroups)

    - #### getColGroups

          public List<ColGroup> getColGroups()

    - #### decompress

          public MatrixBlock decompress()

      Decompress block.
      Specified by:
        `decompress` in class `AbstractCompressedMatrixBlock`
      Returns:
        a new uncompressed matrix block containing the contents of this block

    - #### decompress

          public MatrixBlock decompress(int k)

      Decompress block.
      Parameters:
        `k` - degree of parallelism
      Returns:
        a new uncompressed matrix block containing the contents of this block

    - #### estimateCompressedSizeInMemory

          public long estimateCompressedSizeInMemory()

      Obtain an upper bound on the memory used to store the compressed block.
      Returns:
        an upper bound on the memory used to store this compressed block considering class overhead.

    - #### baseSizeInMemory

          public static long baseSizeInMemory()

    - #### quickGetValue

          public double quickGetValue(int r,
                                      int c)

      Overrides:
        `quickGetValue` in class `MatrixBlock`

    - #### getExactSizeOnDisk

          public long getExactSizeOnDisk()

      Description copied from class: `MatrixBlock`
      NOTE: The used estimates must be kept consistent with the respective write functions.
      Overrides:
        `getExactSizeOnDisk` in class `MatrixBlock`
      Returns:
        exact size on disk

    - #### readFields

          public void readFields(DataInput in)
                          throws IOException

      Specified by:
        `readFields` in interface `org.apache.hadoop.io.Writable`
      Overrides:
        `readFields` in class `MatrixBlock`
      Throws:
        `IOException`

    - #### write

          public void write(DataOutput out)
                     throws IOException

      Specified by:
        `write` in interface `org.apache.hadoop.io.Writable`
      Overrides:
        `write` in class `MatrixBlock`
      Throws:
        `IOException`

    - #### readExternal

          public void readExternal(ObjectInput is)
                            throws IOException

      Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd de-serialization.
      Specified by:
        `readExternal` in interface `Externalizable`
      Overrides:
        `readExternal` in class `MatrixBlock`
      Parameters:
        `is` - object input
      Throws:
        `IOException` - if IOException occurs

    - #### writeExternal

          public void writeExternal(ObjectOutput os)
                             throws IOException

      Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd serialization.
      Specified by:
        `writeExternal` in interface `Externalizable`
      Overrides:
        `writeExternal` in class `MatrixBlock`
      Parameters:
        `os` - object output
      Throws:
        `IOException` - if IOException occurs

    - #### getIterator

          public Iterator<IJV> getIterator(int rl,
                                           int ru,
                                           boolean inclZeros)

    - #### getIterator

          public Iterator<IJV> getIterator(int rl,
                                           int ru,
                                           int cgl,
                                           int cgu,
                                           boolean inclZeros)

    - #### getDenseRowIterator

          public Iterator<double[]> getDenseRowIterator(int rl,
                                                        int ru)

    - #### getSparseRowIterator

          public Iterator<SparseRow> getSparseRowIterator(int rl,
                                                          int ru)

    - #### countNonZerosPerRow

          public int[] countNonZerosPerRow(int rl,
                                           int ru)

    - #### scalarOperations

          public MatrixBlock scalarOperations(ScalarOperator sop,
                                              MatrixValue result)

      Overrides:
        `scalarOperations` in class `MatrixBlock`

    - #### append

          public MatrixBlock append(MatrixBlock that,
                                    MatrixBlock ret)

      Overrides:
        `append` in class `MatrixBlock`

    - #### chainMatrixMultOperations

          public MatrixBlock chainMatrixMultOperations(MatrixBlock v,
                                                       MatrixBlock w,
                                                       MatrixBlock out,
                                                       MapMultChain.ChainType ctype)

      Overrides:
        `chainMatrixMultOperations` in class `MatrixBlock`

    - #### chainMatrixMultOperations

          public MatrixBlock chainMatrixMultOperations(MatrixBlock v,
                                                       MatrixBlock w,
                                                       MatrixBlock out,
                                                       MapMultChain.ChainType ctype,
                                                       int k)

      Overrides:
        `chainMatrixMultOperations` in class `MatrixBlock`

    - #### aggregateBinaryOperations

          public MatrixBlock aggregateBinaryOperations(MatrixBlock m1,
                                                       MatrixBlock m2,
                                                       MatrixBlock ret,
                                                       AggregateBinaryOperator op)

      Overrides:
        `aggregateBinaryOperations` in class `MatrixBlock`

    - #### aggregateUnaryOperations

          public MatrixBlock aggregateUnaryOperations(AggregateUnaryOperator op,
                                                      MatrixValue result,
                                                      int blen,
                                                      MatrixIndexes indexesIn,
                                                      boolean inCP)

      Overrides:
        `aggregateUnaryOperations` in class `MatrixBlock`

    - #### aggregateUnaryOperations

          public MatrixBlock aggregateUnaryOperations(AggregateUnaryOperator op,
                                                      MatrixValue result,
                                                      int blen,
                                                      MatrixIndexes indexesIn)

      Overrides:
        `aggregateUnaryOperations` in class `MatrixBlock`

    - #### transposeSelfMatrixMultOperations

          public MatrixBlock transposeSelfMatrixMultOperations(MatrixBlock out,
                                                               MMTSJ.MMTSJType tstype)

      Overrides:
        `transposeSelfMatrixMultOperations` in class `MatrixBlock`

    - #### transposeSelfMatrixMultOperations

          public MatrixBlock transposeSelfMatrixMultOperations(MatrixBlock out,
                                                               MMTSJ.MMTSJType tstype,
                                                               int k)

      Overrides:
        `transposeSelfMatrixMultOperations` in class `MatrixBlock`

    - #### hasUncompressedColGroup

          public boolean hasUncompressedColGroup()

    - #### toString

          public String toString()

      Overrides:
        `toString` in class `MatrixBlock`

Copyright © 2020 The Apache Software Foundation. All rights reserved.