org.apache.sysds.runtime.compress.estim

## Class CompressedSizeEstimator

- java.lang.Object
- - org.apache.sysds.runtime.compress.estim.CompressedSizeEstimator

- Direct Known Subclasses:
    CompressedSizeEstimatorExact, CompressedSizeEstimatorSample

      public abstract class CompressedSizeEstimator
      extends Object

  Main abstract class for estimating size of compressions on columns.

- - ### Method Summary

    All Methods Static Methods Instance Methods Abstract Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `List<CompressedSizeInfoColGroup>` | `computeCompressedSizeInfos(Collection<int[]> columnLists)` Compression Size info from list of specified columns
    `List<CompressedSizeInfoColGroup>` | `computeCompressedSizeInfos(Collection<int[]> columnLists, int k)` Multi threaded version of extracting Compression Size info from list of specified columns
    `CompressedSizeInfo` | `computeCompressedSizeInfos(int k)` Multi threaded version of extracting Compression Size info
    `CompressedSizeInfoColGroup` | `estimateCompressedColGroupSize()` Method used for compressing into one type of colGroup
    `EstimationFactors` | `estimateCompressedColGroupSize(ABitmap ubm, int[] colIndexes)` Method used to extract the CompressedSizeEstimationFactors from an constructed UncompressedBitmap.
    `static EstimationFactors` | `estimateCompressedColGroupSize(ABitmap ubm, int[] colIndexes, int nrRows, CompressionSettings cs)`
    `CompressedSizeInfoColGroup` | `estimateCompressedColGroupSize(int[] colIndexes)` Method for extracting Compressed Size Info of specified columns, together in a single ColGroup
    `abstract CompressedSizeInfoColGroup` | `estimateCompressedColGroupSize(int[] colIndexes, int nrUniqueUpperBound)` A method to extract the Compressed Size Info for a given list of columns, This method further limits the estimated number of unique values, since in some cases the estimated number of uniques is estimated higher than the number estimated in sub groups of the given colIndexes.
    `CompressedSizeInfoColGroup` | `estimateJoinCompressedSize(CompressedSizeInfoColGroup g1, CompressedSizeInfoColGroup g2)` Join two analyzed column groups together.
    `int` | `getNumColumns()`
    `int` | `getNumRows()`
    `String` | `toString()`

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, wait, wait, wait`

- - ### Method Detail

    - #### getNumRows

          public int getNumRows()

    - #### getNumColumns

          public int getNumColumns()

    - #### computeCompressedSizeInfos

          public CompressedSizeInfo computeCompressedSizeInfos(int k)

      Multi threaded version of extracting Compression Size info
      Parameters:
        `k` - The concurrency degree.
      Returns:
        The Compression Size info of each Column compressed isolated.

    - #### computeCompressedSizeInfos

          public List<CompressedSizeInfoColGroup> computeCompressedSizeInfos(Collection<int[]> columnLists,
                                                                             int k)

      Multi threaded version of extracting Compression Size info from list of specified columns
      Parameters:
        `columnLists` - The specified columns to extract.
        `k` - The parallelization degree
      Returns:
        The Compression information from the specified column groups.

    - #### computeCompressedSizeInfos

          public List<CompressedSizeInfoColGroup> computeCompressedSizeInfos(Collection<int[]> columnLists)

      Compression Size info from list of specified columns
      Parameters:
        `columnLists` - The specified columns to extract.
      Returns:
        The Compression information from the specified column groups.

    - #### estimateCompressedColGroupSize

          public CompressedSizeInfoColGroup estimateCompressedColGroupSize()

      Method used for compressing into one type of colGroup
      Returns:
        CompressedSizeInfo on a compressed colGroup compressing the entire matrix into a single colGroup type.

    - #### estimateCompressedColGroupSize

          public CompressedSizeInfoColGroup estimateCompressedColGroupSize(int[] colIndexes)

      Method for extracting Compressed Size Info of specified columns, together in a single ColGroup
      Parameters:
        `colIndexes` - The columns to group together inside a ColGroup
      Returns:
        The CompressedSizeInformation associated with the selected ColGroups.

    - #### estimateCompressedColGroupSize

          public abstract CompressedSizeInfoColGroup estimateCompressedColGroupSize(int[] colIndexes,
                                                                                    int nrUniqueUpperBound)

      A method to extract the Compressed Size Info for a given list of columns, This method further limits the estimated number of unique values, since in some cases the estimated number of uniques is estimated higher than the number estimated in sub groups of the given colIndexes.
      Parameters:
        `colIndexes` - The columns to extract compression information from
        `nrUniqueUpperBound` - The upper bound of unique elements allowed in the estimate, can be calculated from the number of unique elements estimated in sub columns multiplied together. This is flexible in the sense that if the sample is small then this unique can be manually edited like in CoCodeCostMatrixMult.
      Returns:
        The CompressedSizeInfoColGroup fro the given column indexes.

    - #### estimateJoinCompressedSize

          public CompressedSizeInfoColGroup estimateJoinCompressedSize(CompressedSizeInfoColGroup g1,
                                                                       CompressedSizeInfoColGroup g2)

      Join two analyzed column groups together. without materializing the dictionaries of either side. If either side was constructed without analysis then fall back to default materialization of double arrays.
      Parameters:
        `g1` - First group
        `g2` - Second group
      Returns:
        A joined compressed size estimation for the group.

    - #### estimateCompressedColGroupSize

          public EstimationFactors estimateCompressedColGroupSize(ABitmap ubm,
                                                                  int[] colIndexes)

      Method used to extract the CompressedSizeEstimationFactors from an constructed UncompressedBitmap. Note this method works both for the sample based estimator and the exact estimator, since the bitmap, can be extracted from a sample or from the entire dataset.
      Parameters:
        `ubm` - The UncompressedBitmap, either extracted from a sample or from the entire dataset
        `colIndexes` - The columns that is compressed together.
      Returns:
        The size factors estimated from the Bit Map.

    - #### estimateCompressedColGroupSize

          public static EstimationFactors estimateCompressedColGroupSize(ABitmap ubm,
                                                                         int[] colIndexes,
                                                                         int nrRows,
                                                                         CompressionSettings cs)

    - #### toString

          public String toString()

      Overrides:
        `toString` in class `Object`

Copyright © 2021 The Apache Software Foundation. All rights reserved.