Package org.apache.sysds.runtime.matrix.data

## Class LibMatrixAgg

- java.lang.Object
- - org.apache.sysds.runtime.matrix.data.LibMatrixAgg

-     public class LibMatrixAgg
      extends Object

  MB: Library for matrix aggregations including ak+, uak+ for all combinations of dense and sparse representations, and corrections. Those are performance-critical operations because they are used on combiners/reducers of important operations like tsmm, mvmult, indexing, but also basic sum/min/max/mean, row*, col*, etc. Specific handling is especially required for all non sparse-safe operations in order to prevent unnecessary worse asymptotic behavior. This library currently covers the following opcodes: ak+, uak+, uark+, uack+, uasqk+, uarsqk+, uacsqk+, uamin, uarmin, uacmin, uamax, uarmax, uacmax, ua*, uamean, uarmean, uacmean, uavar, uarvar, uacvar, uarimax, uaktrace, cumk+, cummin, cummax, cum*, tak+, cm, cov TODO next opcode extensions: a+, colindexmax

- - ### Method Summary

    All Methods Static Methods Concrete Methods
    Modifier and Type | Method | Description
    --- | --- | ---
    `static void` | `aggregateBinaryMatrix​(MatrixBlock in, MatrixBlock aggVal, MatrixBlock aggCorr, boolean deep)` | Core incremental matrix aggregate (ak+) as used in mapmult, tsmm, cpmm, etc.
    `static void` | `aggregateBinaryMatrix​(MatrixBlock in, MatrixBlock aggVal, AggregateOperator aop)` | Core incremental matrix aggregate (ak+) as used for uack+ and acrk+.
    `static CM_COV_Object` | `aggregateCmCov​(MatrixBlock in1, MatrixBlock in2, MatrixBlock in3, ValueFunction fn)` | Single threaded Covariance and Central Moment operations CM = Central Moment COV = Covariance
    `static CM_COV_Object` | `aggregateCmCov​(MatrixBlock in1, MatrixBlock in2, MatrixBlock in3, ValueFunction fn, int k)` | Multi threaded Covariance and Central Moment operations CM = Central Moment COV = Covariance
    `static MatrixBlock` | `aggregateTernary​(MatrixBlock in1, MatrixBlock in2, MatrixBlock in3, MatrixBlock ret, AggregateTernaryOperator op)` |
    `static MatrixBlock` | `aggregateTernary​(MatrixBlock in1, MatrixBlock in2, MatrixBlock in3, MatrixBlock ret, AggregateTernaryOperator op, int k)` |
    `static void` | `aggregateUnaryMatrix​(MatrixBlock in, MatrixBlock out, AggregateUnaryOperator uaop)` |
    `static void` | `aggregateUnaryMatrix​(MatrixBlock in, MatrixBlock out, AggregateUnaryOperator uaop, int k)` |
    `static void` | `countAgg​(double[] a, int[] c, int[] aix, int ai, int len)` |
    `static void` | `countAgg​(double[] a, int[] c, int ai, int len)` |
    `static MatrixBlock` | `cumaggregateUnaryMatrix​(MatrixBlock in, MatrixBlock out, UnaryOperator uop)` |
    `static MatrixBlock` | `cumaggregateUnaryMatrix​(MatrixBlock in, MatrixBlock out, UnaryOperator uop, double[] agg)` |
    `static MatrixBlock` | `cumaggregateUnaryMatrix​(MatrixBlock in, MatrixBlock out, UnaryOperator uop, int k)` |
    `static void` | `groupedAggregate​(MatrixBlock groups, MatrixBlock target, MatrixBlock weights, MatrixBlock result, int numGroups, Operator op)` |
    `static void` | `groupedAggregate​(MatrixBlock groups, MatrixBlock target, MatrixBlock weights, MatrixBlock result, int numGroups, Operator op, int k)` |
    `static boolean` | `isSupportedUnaryAggregateOperator​(AggregateUnaryOperator op)` |
    `static boolean` | `isSupportedUnaryOperator​(UnaryOperator op)` |
    `static void` | `recomputeIndexes​(MatrixBlock out, AggregateUnaryOperator op, int blen, MatrixIndexes ix)` | Recompute outputs (e.g., maxindex or minindex) according to block indexes from MR.
    `static boolean` | `satisfiesMultiThreadingConstraints​(MatrixBlock in, int k)` |
    `static boolean` | `satisfiesMultiThreadingConstraints​(MatrixBlock in, MatrixBlock out, AggregateUnaryOperator uaop, int k)` |

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Method Detail

    - #### aggregateBinaryMatrix

          public static void aggregateBinaryMatrix​(MatrixBlock in,
                                                   MatrixBlock aggVal,
                                                   MatrixBlock aggCorr,
                                                   boolean deep)

      Core incremental matrix aggregate (ak+) as used in mapmult, tsmm, cpmm, etc. Note that we try to keep the current aggVal and aggCorr in dense format in order to allow efficient access according to the dense/sparse input.
      Parameters:
        `in` - input matrix
        `aggVal` - current aggregate values (in/out)
        `aggCorr` - current aggregate correction (in/out)
        `deep` - deep copy flag

    - #### aggregateBinaryMatrix

          public static void aggregateBinaryMatrix​(MatrixBlock in,
                                                   MatrixBlock aggVal,
                                                   AggregateOperator aop)

      Core incremental matrix aggregate (ak+) as used for uack+ and acrk+. Embedded correction values. DOES NOT EVALUATE SPARSITY SINCE IT IS USED IN INCREMENTAL AGGREGATION
      Parameters:
        `in` - matrix block
        `aggVal` - aggregate operator
        `aop` - aggregate operator

    - #### aggregateUnaryMatrix

          public static void aggregateUnaryMatrix​(MatrixBlock in,
                                                  MatrixBlock out,
                                                  AggregateUnaryOperator uaop)

    - #### aggregateUnaryMatrix

          public static void aggregateUnaryMatrix​(MatrixBlock in,
                                                  MatrixBlock out,
                                                  AggregateUnaryOperator uaop,
                                                  int k)

    - #### cumaggregateUnaryMatrix

          public static MatrixBlock cumaggregateUnaryMatrix​(MatrixBlock in,
                                                            MatrixBlock out,
                                                            UnaryOperator uop)

    - #### cumaggregateUnaryMatrix

          public static MatrixBlock cumaggregateUnaryMatrix​(MatrixBlock in,
                                                            MatrixBlock out,
                                                            UnaryOperator uop,
                                                            double[] agg)

    - #### cumaggregateUnaryMatrix

          public static MatrixBlock cumaggregateUnaryMatrix​(MatrixBlock in,
                                                            MatrixBlock out,
                                                            UnaryOperator uop,
                                                            int k)

    - #### aggregateCmCov

          public static CM_COV_Object aggregateCmCov​(MatrixBlock in1,
                                                     MatrixBlock in2,
                                                     MatrixBlock in3,
                                                     ValueFunction fn)

      Single threaded Covariance and Central Moment operations CM = Central Moment COV = Covariance
      Parameters:
        `in1` - Main input matrix
        `in2` - Second input matrix
        `in3` - Third input matrix (not output since output is returned)
        `fn` - Value function to apply
      Returns:
        Central Moment or Covariance object

    - #### aggregateCmCov

          public static CM_COV_Object aggregateCmCov​(MatrixBlock in1,
                                                     MatrixBlock in2,
                                                     MatrixBlock in3,
                                                     ValueFunction fn,
                                                     int k)

      Multi threaded Covariance and Central Moment operations CM = Central Moment COV = Covariance
      Parameters:
        `in1` - Main input matrix
        `in2` - Second input matrix
        `in3` - Third input matrix (not output since output is returned)
        `fn` - Value function to apply
        `k` - Parallelization degree
      Returns:
        Central Moment or Covariance object

    - #### aggregateTernary

          public static MatrixBlock aggregateTernary​(MatrixBlock in1,
                                                     MatrixBlock in2,
                                                     MatrixBlock in3,
                                                     MatrixBlock ret,
                                                     AggregateTernaryOperator op)

    - #### aggregateTernary

          public static MatrixBlock aggregateTernary​(MatrixBlock in1,
                                                     MatrixBlock in2,
                                                     MatrixBlock in3,
                                                     MatrixBlock ret,
                                                     AggregateTernaryOperator op,
                                                     int k)

    - #### groupedAggregate

          public static void groupedAggregate​(MatrixBlock groups,
                                              MatrixBlock target,
                                              MatrixBlock weights,
                                              MatrixBlock result,
                                              int numGroups,
                                              Operator op)

    - #### groupedAggregate

          public static void groupedAggregate​(MatrixBlock groups,
                                              MatrixBlock target,
                                              MatrixBlock weights,
                                              MatrixBlock result,
                                              int numGroups,
                                              Operator op,
                                              int k)

    - #### isSupportedUnaryAggregateOperator

          public static boolean isSupportedUnaryAggregateOperator​(AggregateUnaryOperator op)

    - #### isSupportedUnaryOperator

          public static boolean isSupportedUnaryOperator​(UnaryOperator op)

    - #### satisfiesMultiThreadingConstraints

          public static boolean satisfiesMultiThreadingConstraints​(MatrixBlock in,
                                                                   MatrixBlock out,
                                                                   AggregateUnaryOperator uaop,
                                                                   int k)

    - #### satisfiesMultiThreadingConstraints

          public static boolean satisfiesMultiThreadingConstraints​(MatrixBlock in,
                                                                   int k)

    - #### recomputeIndexes

          public static void recomputeIndexes​(MatrixBlock out,
                                              AggregateUnaryOperator op,
                                              int blen,
                                              MatrixIndexes ix)

      Recompute outputs (e.g., maxindex or minindex) according to block indexes from MR. TODO: this should not be part of block operations but of the MR instruction.
      Parameters:
        `out` - matrix block
        `op` - aggregate unary operator
        `blen` - number of rows/cols in a block
        `ix` - matrix indexes

    - #### countAgg

          public static void countAgg​(double[] a,
                                      int[] c,
                                      int[] aix,
                                      int ai,
                                      int len)

    - #### countAgg

          public static void countAgg​(double[] a,
                                      int[] c,
                                      int ai,
                                      int len)