org.apache.sysds.utils

## Class Statistics

- java.lang.Object
- - org.apache.sysds.utils.Statistics

-     public class Statistics
      extends Object

  This class captures all statistics.

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static long` | `allocateDoubleArrTime`
    `static boolean` | `allowWorkerStatistics`
    `static long` | `examSparsityTime`
    `static long` | `nativeConv2dBwdDataTime`
    `static long` | `nativeConv2dBwdFilterTime`
    `static long` | `nativeConv2dTime`
    `static long` | `nativeLibMatrixMultTime`
    `static LongAdder` | `numNativeConv2dBwdDataCalls`
    `static LongAdder` | `numNativeConv2dBwdFilterCalls`
    `static LongAdder` | `numNativeConv2dCalls`
    `static LongAdder` | `numNativeLibMatrixMultCalls`
    `static LongAdder` | `numNativeSparseConv2dBwdDataCalls`
    `static LongAdder` | `numNativeSparseConv2dBwdFilterCalls`
    `static LongAdder` | `numNativeSparseConv2dCalls`
    `static long` | `recomputeNNZTime`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `Statistics()`

  - ### Method Summary

    All Methods Static Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `static void` | `accFedPSCommunicationTime(long t)`
    `static void` | `accFedPSDataPartitioningTime(long t)`
    `static void` | `accFedPSGradientWeightingTime(long t)`
    `static void` | `accFedPSWorkerComputing(long t)`
    `static void` | `accPSAggregationTime(long t)`
    `static void` | `accPSBatchIndexingTime(long t)`
    `static void` | `accPSExecutionTime(long n)`
    `static void` | `accPSGradientComputeTime(long t)`
    `static void` | `accPSLocalModelUpdateTime(long t)`
    `static void` | `accPSModelBroadcastTime(long t)`
    `static void` | `accPSRpcRequestTime(long t)`
    `static void` | `accPSSetupTime(long t)`
    `static void` | `accPSValidationTime(long t)`
    `static void` | `accSparkBroadCastTime(long t)`
    `static void` | `accSparkCollectTime(long t)`
    `static void` | `accSparkParallelizeTime(long t)`
    `static void` | `addCPMemObject(int hash, double sizeof)`
    `static boolean` | `createdSparkContext()`
    `static void` | `decrementNoOfExecutedSPInst()`
    `static String` | `display()` Returns statistics of the DML program that was recently completed as a string
    `static String` | `display(int maxHeavyHitters)` Returns statistics as a string
    `static long` | `getAsyncBroadcastCount()`
    `static long` | `getAsyncPrefetchCount()`
    `static long` | `getCodegenClassCompile()`
    `static long` | `getCodegenClassCompileTime()`
    `static long` | `getCodegenCompileTime()`
    `static long` | `getCodegenCPlanCompile()`
    `static long` | `getCodegenDAGCompile()`
    `static long` | `getCodegenEnumAll()`
    `static long` | `getCodegenEnumAllP()`
    `static long` | `getCodegenEnumEval()`
    `static long` | `getCodegenEnumEvalP()`
    `static long` | `getCodegenOpCacheHits()`
    `static long` | `getCodegenOpCacheTotal()`
    `static long` | `getCodegenPlanCacheHits()`
    `static long` | `getCodegenPlanCacheTotal()`
    `static long` | `getCompileTime()`
    `static String` | `getCPHeavyHitterCode(Instruction inst)`
    `static long` | `getCPHeavyHitterCount(String opcode)`
    `static Set<String>` | `getCPHeavyHitterOpCodes()`
    `static String` | `getCPHeavyHittersMem(int num)`
    `static long` | `getFunRecompiles()`
    `static long` | `getFunRecompileTime()`
    `static String` | `getHeavyHitters(int num)` Obtain a string tabular representation of the heavy hitter instructions that displays the time, instruction count, and optionally GPU stats about each instruction.
    `static HashMap<String,org.apache.commons.lang3.tuple.Pair<Long,Double>>` | `getHeavyHittersHashMap()`
    `static long` | `getHopRecompiledPredDAGs()`
    `static long` | `getHopRecompiledSBDAGs()`
    `static long` | `getHopRecompileTime()`
    `static long` | `getJITCompileTime()` Returns the total time of asynchronous JIT compilation in milliseconds.
    `static long` | `getJVMgcCount()`
    `static long` | `getJVMgcTime()`
    `static long` | `getNoOfCompiledSPInst()`
    `static long` | `getNoOfExecutedSPInst()`
    `static long` | `getNumPinnedObjects()`
    `static long` | `getParforInitTime()`
    `static long` | `getParforMergeTime()`
    `static long` | `getParforOptCount()`
    `static long` | `getParforOptTime()`
    `static double` | `getPSExecutionTime()`
    `static Timing` | `getPSExecutionTimer()`
    `static double` | `getPSValidationTime()`
    `static long` | `getRunTime()` Returns the total time of run in nanoseconds.
    `static double` | `getSizeofPinnedObjects()`
    `static long` | `getTotalLix()`
    `static long` | `getTotalLixUIP()`
    `static long` | `getTotalUIPVar()`
    `static void` | `incFederated(FederatedRequest.RequestType rqt)`
    `static void` | `incrementCodegenClassCompile()`
    `static void` | `incrementCodegenClassCompileTime(long delta)`
    `static void` | `incrementCodegenCompileTime(long delta)`
    `static void` | `incrementCodegenCPlanCompile(long delta)`
    `static void` | `incrementCodegenDAGCompile()`
    `static void` | `incrementCodegenEnumAll(long delta)`
    `static void` | `incrementCodegenEnumAllP(long delta)`
    `static void` | `incrementCodegenEnumEval(long delta)`
    `static void` | `incrementCodegenEnumEvalP(long delta)`
    `static void` | `incrementCodegenOpCacheHits()`
    `static void` | `incrementCodegenOpCacheTotal()`
    `static void` | `incrementCodegenPlanCacheHits()`
    `static void` | `incrementCodegenPlanCacheTotal()`
    `static void` | `incrementFunRecompiles()`
    `static void` | `incrementFunRecompileTime(long delta)`
    `static void` | `incrementHOPRecompilePred()`
    `static void` | `incrementHOPRecompilePred(long delta)`
    `static void` | `incrementHOPRecompileSB()`
    `static void` | `incrementHOPRecompileSB(long delta)`
    `static void` | `incrementHOPRecompileTime(long delta)`
    `static void` | `incrementJITCompileTime(long time)`
    `static void` | `incrementJVMgcCount(long delta)`
    `static void` | `incrementJVMgcTime(long time)`
    `static void` | `incrementNativeFailuresCounter()`
    `static void` | `incrementNoOfCompiledSPInst()`
    `static void` | `incrementNoOfExecutedSPInst()`
    `static void` | `incrementParForInitTime(long time)`
    `static void` | `incrementParForMergeTime(long time)`
    `static void` | `incrementParForOptimCount()`
    `static void` | `incrementParForOptimTime(long time)`
    `static void` | `incrementTotalLix()`
    `static void` | `incrementTotalLixUIP()`
    `static void` | `incrementTotalUIPVar()`
    `static void` | `incSparkAsyncBroadcastCount(long c)`
    `static void` | `incSparkAsyncPrefetchCount(long c)`
    `static void` | `incSparkBroadcastCount(long c)`
    `static void` | `incSparkCollectCount(long c)`
    `static void` | `incSparkParallelizeCount(long c)`
    `static void` | `incWorkerNumber()`
    `static void` | `incWorkerNumber(long n)`
    `static void` | `maintainCPFuncCallStats(String instName)`
    `static void` | `maintainCPHeavyHitters(String instName, long timeNanos)` "Maintains" or adds time to per instruction/op timers, also increments associated count
    `static void` | `maintainCPHeavyHittersMem(String name, double sizeof)` Helper method which keeps track of the heaviest weight objects (by total memory used) throughout execution of the program.
    `static void` | `removeCPMemObject(int hash)` Helper method to remove a memory object which has become unpinned
    `static void` | `reset()`
    `static void` | `resetCPHeavyHitters()`
    `static void` | `resetJITCompileTime()`
    `static void` | `resetJVMgcCount()`
    `static void` | `resetJVMgcTime()`
    `static void` | `resetNoOfCompiledJobs(int count)`
    `static void` | `resetNoOfExecutedJobs()`
    `static void` | `setSparkCtxCreateTime(long ns)`
    `static void` | `startCompileTimer()`
    `static void` | `startRunTimer()` Starts the timer, should be invoked immediately before invoking Program.execute()
    `static void` | `stopCompileTimer()`
    `static void` | `stopRunTimer()` Stops the timer, should be invoked immediately after invoking Program.execute()
    `static String[]` | `wrap(String str, int wrapLength)`

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### numNativeLibMatrixMultCalls

          public static LongAdder numNativeLibMatrixMultCalls

    - #### numNativeConv2dCalls

          public static LongAdder numNativeConv2dCalls

    - #### numNativeConv2dBwdDataCalls

          public static LongAdder numNativeConv2dBwdDataCalls

    - #### numNativeConv2dBwdFilterCalls

          public static LongAdder numNativeConv2dBwdFilterCalls

    - #### numNativeSparseConv2dCalls

          public static LongAdder numNativeSparseConv2dCalls

    - #### numNativeSparseConv2dBwdFilterCalls

          public static LongAdder numNativeSparseConv2dBwdFilterCalls

    - #### numNativeSparseConv2dBwdDataCalls

          public static LongAdder numNativeSparseConv2dBwdDataCalls

    - #### nativeLibMatrixMultTime

          public static long nativeLibMatrixMultTime

    - #### nativeConv2dTime

          public static long nativeConv2dTime

    - #### nativeConv2dBwdDataTime

          public static long nativeConv2dBwdDataTime

    - #### nativeConv2dBwdFilterTime

          public static long nativeConv2dBwdFilterTime

    - #### recomputeNNZTime

          public static long recomputeNNZTime

    - #### examSparsityTime

          public static long examSparsityTime

    - #### allocateDoubleArrTime

          public static long allocateDoubleArrTime

    - #### allowWorkerStatistics

          public static boolean allowWorkerStatistics

  - ### Constructor Detail

    - #### Statistics

          public Statistics()

  - ### Method Detail

    - #### incrementNativeFailuresCounter

          public static void incrementNativeFailuresCounter()

    - #### getNoOfExecutedSPInst

          public static long getNoOfExecutedSPInst()

    - #### incrementNoOfExecutedSPInst

          public static void incrementNoOfExecutedSPInst()

    - #### decrementNoOfExecutedSPInst

          public static void decrementNoOfExecutedSPInst()

    - #### getNoOfCompiledSPInst

          public static long getNoOfCompiledSPInst()

    - #### incrementNoOfCompiledSPInst

          public static void incrementNoOfCompiledSPInst()

    - #### createdSparkContext

          public static boolean createdSparkContext()

    - #### getTotalUIPVar

          public static long getTotalUIPVar()

    - #### incrementTotalUIPVar

          public static void incrementTotalUIPVar()

    - #### getTotalLixUIP

          public static long getTotalLixUIP()

    - #### incrementTotalLixUIP

          public static void incrementTotalLixUIP()

    - #### getTotalLix

          public static long getTotalLix()

    - #### incrementTotalLix

          public static void incrementTotalLix()

    - #### resetNoOfCompiledJobs

          public static void resetNoOfCompiledJobs(int count)

    - #### resetNoOfExecutedJobs

          public static void resetNoOfExecutedJobs()

    - #### incrementJITCompileTime

          public static void incrementJITCompileTime(long time)

    - #### incrementJVMgcTime

          public static void incrementJVMgcTime(long time)

    - #### incrementJVMgcCount

          public static void incrementJVMgcCount(long delta)

    - #### incrementHOPRecompileTime

          public static void incrementHOPRecompileTime(long delta)

    - #### incrementHOPRecompilePred

          public static void incrementHOPRecompilePred()

    - #### incrementHOPRecompilePred

          public static void incrementHOPRecompilePred(long delta)

    - #### incrementHOPRecompileSB

          public static void incrementHOPRecompileSB()

    - #### incrementHOPRecompileSB

          public static void incrementHOPRecompileSB(long delta)

    - #### incrementCodegenDAGCompile

          public static void incrementCodegenDAGCompile()

    - #### incrementCodegenCPlanCompile

          public static void incrementCodegenCPlanCompile(long delta)

    - #### incrementCodegenEnumAll

          public static void incrementCodegenEnumAll(long delta)

    - #### incrementCodegenEnumAllP

          public static void incrementCodegenEnumAllP(long delta)

    - #### incrementCodegenEnumEval

          public static void incrementCodegenEnumEval(long delta)

    - #### incrementCodegenEnumEvalP

          public static void incrementCodegenEnumEvalP(long delta)

    - #### incrementCodegenClassCompile

          public static void incrementCodegenClassCompile()

    - #### incrementCodegenCompileTime

          public static void incrementCodegenCompileTime(long delta)

    - #### incrementCodegenClassCompileTime

          public static void incrementCodegenClassCompileTime(long delta)

    - #### incrementCodegenOpCacheHits

          public static void incrementCodegenOpCacheHits()

    - #### incrementCodegenOpCacheTotal

          public static void incrementCodegenOpCacheTotal()

    - #### incrementCodegenPlanCacheHits

          public static void incrementCodegenPlanCacheHits()

    - #### incrementCodegenPlanCacheTotal

          public static void incrementCodegenPlanCacheTotal()

    - #### getCodegenDAGCompile

          public static long getCodegenDAGCompile()

    - #### getCodegenCPlanCompile

          public static long getCodegenCPlanCompile()

    - #### getCodegenEnumAll

          public static long getCodegenEnumAll()

    - #### getCodegenEnumAllP

          public static long getCodegenEnumAllP()

    - #### getCodegenEnumEval

          public static long getCodegenEnumEval()

    - #### getCodegenEnumEvalP

          public static long getCodegenEnumEvalP()

    - #### getCodegenClassCompile

          public static long getCodegenClassCompile()

    - #### getCodegenCompileTime

          public static long getCodegenCompileTime()

    - #### getCodegenClassCompileTime

          public static long getCodegenClassCompileTime()

    - #### getCodegenOpCacheHits

          public static long getCodegenOpCacheHits()

    - #### getCodegenOpCacheTotal

          public static long getCodegenOpCacheTotal()

    - #### getCodegenPlanCacheHits

          public static long getCodegenPlanCacheHits()

    - #### getCodegenPlanCacheTotal

          public static long getCodegenPlanCacheTotal()

    - #### incrementFunRecompileTime

          public static void incrementFunRecompileTime(long delta)

    - #### incrementFunRecompiles

          public static void incrementFunRecompiles()

    - #### incrementParForOptimCount

          public static void incrementParForOptimCount()

    - #### incrementParForOptimTime

          public static void incrementParForOptimTime(long time)

    - #### incrementParForInitTime

          public static void incrementParForInitTime(long time)

    - #### incrementParForMergeTime

          public static void incrementParForMergeTime(long time)

    - #### incFederated

          public static void incFederated(FederatedRequest.RequestType rqt)

    - #### startCompileTimer

          public static void startCompileTimer()

    - #### stopCompileTimer

          public static void stopCompileTimer()

    - #### getCompileTime

          public static long getCompileTime()

    - #### startRunTimer

          public static void startRunTimer()

      Starts the timer, should be invoked immediately before invoking Program.execute()

    - #### stopRunTimer

          public static void stopRunTimer()

      Stops the timer, should be invoked immediately after invoking Program.execute()

    - #### getRunTime

          public static long getRunTime()

      Returns the total time of run in nanoseconds.
      Returns:
        run time in nanoseconds

    - #### reset

          public static void reset()

    - #### resetJITCompileTime

          public static void resetJITCompileTime()

    - #### resetJVMgcTime

          public static void resetJVMgcTime()

    - #### resetJVMgcCount

          public static void resetJVMgcCount()

    - #### resetCPHeavyHitters

          public static void resetCPHeavyHitters()

    - #### setSparkCtxCreateTime

          public static void setSparkCtxCreateTime(long ns)

    - #### accSparkParallelizeTime

          public static void accSparkParallelizeTime(long t)

    - #### incSparkParallelizeCount

          public static void incSparkParallelizeCount(long c)

    - #### accSparkCollectTime

          public static void accSparkCollectTime(long t)

    - #### incSparkCollectCount

          public static void incSparkCollectCount(long c)

    - #### accSparkBroadCastTime

          public static void accSparkBroadCastTime(long t)

    - #### incSparkBroadcastCount

          public static void incSparkBroadcastCount(long c)

    - #### incSparkAsyncPrefetchCount

          public static void incSparkAsyncPrefetchCount(long c)

    - #### incSparkAsyncBroadcastCount

          public static void incSparkAsyncBroadcastCount(long c)

    - #### incWorkerNumber

          public static void incWorkerNumber()

    - #### incWorkerNumber

          public static void incWorkerNumber(long n)

    - #### getPSExecutionTimer

          public static Timing getPSExecutionTimer()

    - #### getPSExecutionTime

          public static double getPSExecutionTime()

    - #### accPSExecutionTime

          public static void accPSExecutionTime(long n)

    - #### accPSSetupTime

          public static void accPSSetupTime(long t)

    - #### accPSGradientComputeTime

          public static void accPSGradientComputeTime(long t)

    - #### accPSAggregationTime

          public static void accPSAggregationTime(long t)

    - #### accPSLocalModelUpdateTime

          public static void accPSLocalModelUpdateTime(long t)

    - #### accPSModelBroadcastTime

          public static void accPSModelBroadcastTime(long t)

    - #### accPSBatchIndexingTime

          public static void accPSBatchIndexingTime(long t)

    - #### accPSRpcRequestTime

          public static void accPSRpcRequestTime(long t)

    - #### getPSValidationTime

          public static double getPSValidationTime()

    - #### accPSValidationTime

          public static void accPSValidationTime(long t)

    - #### accFedPSDataPartitioningTime

          public static void accFedPSDataPartitioningTime(long t)

    - #### accFedPSWorkerComputing

          public static void accFedPSWorkerComputing(long t)

    - #### accFedPSGradientWeightingTime

          public static void accFedPSGradientWeightingTime(long t)

    - #### accFedPSCommunicationTime

          public static void accFedPSCommunicationTime(long t)

    - #### getCPHeavyHitterCode

          public static String getCPHeavyHitterCode(Instruction inst)

    - #### addCPMemObject

          public static void addCPMemObject(int hash,
                                            double sizeof)

    - #### removeCPMemObject

          public static void removeCPMemObject(int hash)

      Helper method to remove a memory object which has become unpinned
      Parameters:
        `hash` - hash of data object

    - #### maintainCPHeavyHittersMem

          public static void maintainCPHeavyHittersMem(String name,
                                                       double sizeof)

      Helper method which keeps track of the heaviest weight objects (by total memory used) throughout execution of the program. Only reported if JMLC memory statistics are enabled and finegrained statistics are enabled. We only keep track of the -largest- instance of data associated with a particular string identifier so no need to worry about multiple bindings to the same name
      Parameters:
        `name` - String denoting the variables name
        `sizeof` - objects size (estimated bytes)

    - #### maintainCPHeavyHitters

          public static void maintainCPHeavyHitters(String instName,
                                                    long timeNanos)

      "Maintains" or adds time to per instruction/op timers, also increments associated count
      Parameters:
        `instName` - name of the instruction/op
        `timeNanos` - time in nano seconds

    - #### maintainCPFuncCallStats

          public static void maintainCPFuncCallStats(String instName)

    - #### getCPHeavyHitterOpCodes

          public static Set<String> getCPHeavyHitterOpCodes()

    - #### getCPHeavyHitterCount

          public static long getCPHeavyHitterCount(String opcode)

    - #### getHeavyHittersHashMap

          public static HashMap<String,org.apache.commons.lang3.tuple.Pair<Long,Double>> getHeavyHittersHashMap()

    - #### getHeavyHitters

          public static String getHeavyHitters(int num)

      Obtain a string tabular representation of the heavy hitter instructions that displays the time, instruction count, and optionally GPU stats about each instruction.
      Parameters:
        `num` - the maximum number of heavy hitters to display
      Returns:
        string representing the heavy hitter instructions in tabular format

    - #### getCPHeavyHittersMem

          public static String getCPHeavyHittersMem(int num)

    - #### getJITCompileTime

          public static long getJITCompileTime()

      Returns the total time of asynchronous JIT compilation in milliseconds.
      Returns:
        JIT compile time

    - #### getJVMgcTime

          public static long getJVMgcTime()

    - #### getJVMgcCount

          public static long getJVMgcCount()

    - #### getHopRecompileTime

          public static long getHopRecompileTime()

    - #### getHopRecompiledPredDAGs

          public static long getHopRecompiledPredDAGs()

    - #### getHopRecompiledSBDAGs

          public static long getHopRecompiledSBDAGs()

    - #### getFunRecompileTime

          public static long getFunRecompileTime()

    - #### getFunRecompiles

          public static long getFunRecompiles()

    - #### getParforOptCount

          public static long getParforOptCount()

    - #### getParforOptTime

          public static long getParforOptTime()

    - #### getParforInitTime

          public static long getParforInitTime()

    - #### getParforMergeTime

          public static long getParforMergeTime()

    - #### getNumPinnedObjects

          public static long getNumPinnedObjects()

    - #### getSizeofPinnedObjects

          public static double getSizeofPinnedObjects()

    - #### getAsyncPrefetchCount

          public static long getAsyncPrefetchCount()

    - #### getAsyncBroadcastCount

          public static long getAsyncBroadcastCount()

    - #### display

          public static String display()

      Returns statistics of the DML program that was recently completed as a string
      Returns:
        statistics as a string

    - #### wrap

          public static String[] wrap(String str,
                                      int wrapLength)

    - #### display

          public static String display(int maxHeavyHitters)

      Returns statistics as a string
      Parameters:
        `maxHeavyHitters` - The maximum number of heavy hitters that are printed
      Returns:
        statistics as string

Copyright © 2021 The Apache Software Foundation. All rights reserved.