Package org.apache.sysds.runtime.controlprogram.parfor.stat

## Class InfrastructureAnalyzer

- java.lang.Object
- - org.apache.sysds.runtime.controlprogram.parfor.stat.InfrastructureAnalyzer

-     public class InfrastructureAnalyzer
      extends Object

  Central place for analyzing and obtaining static infrastructure properties such as memory and number of logical processors.

- - ### Field Summary

    Fields
    Modifier and Type | Field | Description
    --- | --- | ---
    `static long` | `DEFAULT_JVM_SIZE` |
    `static String` | `MR_FRAMEWORK_NAME` |
    `static String` | `MR_JOBTRACKER_ADDRESS` |

  - ### Constructor Summary

    Constructors
    Constructor | Description
    --- | ---
    `InfrastructureAnalyzer()` |

  - ### Method Summary

    All Methods Static Methods Concrete Methods
    Modifier and Type | Method | Description
    --- | --- | ---
    `static long` | `extractMaxMemoryOpt​(String javaOpts)` |
    `static int` | `getCkMaxCP()` | Gets the maximum local parallelism constraint.
    `static int` | `getCkMaxMR()` | Gets the maximum remote parallelism constraint
    `static double` | `getClusterUtilization​(boolean mapOnly)` | Gets the fraction of running map/reduce tasks to existing map/reduce task slots.
    `static long` | `getCmMax()` | Gets the maximum memory constraint [in bytes].
    `static long` | `getHDFSBlockSize()` | Gets the HDFS blocksize of the used cluster in bytes.
    `static long` | `getLocalMaxMemory()` | Gets the maximum memory [in bytes] of the current JVM.
    `static double` | `getLocalMaxMemoryFraction()` |
    `static int` | `getLocalParallelism()` | Gets the number of logical processors of the current node, including hyper-threading if enabled.
    `static int` | `getRemoteParallelMapTasks()` | Gets the number of remote parallel map slots.
    `static int` | `getRemoteParallelNodes()` | Gets the number of cluster nodes (number of tasktrackers).
    `static int` | `getRemoteParallelReduceTasks()` | Gets the total number of remote parallel reduce slots.
    `static boolean` | `isLocalMode()` |
    `static boolean` | `isLocalMode​(org.apache.hadoop.mapred.JobConf job)` |
    `static void` | `setLocalMaxMemory​(long localMem)` |
    `static void` | `setMaxMemoryOpt​(org.apache.hadoop.mapred.JobConf job, String key, long bytes)` |
    `static void` | `setRemoteParallelMapTasks​(int pmap)` |
    `static void` | `setRemoteParallelReduceTasks​(int preduce)` |

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### DEFAULT_JVM_SIZE

          public static final long DEFAULT_JVM_SIZE

      See Also:
        Constant Field Values

    - #### MR_FRAMEWORK_NAME

          public static final String MR_FRAMEWORK_NAME

      See Also:
        Constant Field Values

    - #### MR_JOBTRACKER_ADDRESS

          public static final String MR_JOBTRACKER_ADDRESS

      See Also:
        Constant Field Values

  - ### Constructor Detail

    - #### InfrastructureAnalyzer

          public InfrastructureAnalyzer()

  - ### Method Detail

    - #### getLocalParallelism

          public static int getLocalParallelism()

      Gets the number of logical processors of the current node, including hyper-threading if enabled.
      Returns:
        number of local processors of the current node

    - #### getRemoteParallelNodes

          public static int getRemoteParallelNodes()

      Gets the number of cluster nodes (number of tasktrackers). If multiple tasktracker are started per node, each tasktracker is viewed as individual node.
      Returns:
        number of cluster nodes

    - #### getRemoteParallelMapTasks

          public static int getRemoteParallelMapTasks()

      Gets the number of remote parallel map slots.
      Returns:
        number of remote parallel map tasks

    - #### setRemoteParallelMapTasks

          public static void setRemoteParallelMapTasks​(int pmap)

    - #### getRemoteParallelReduceTasks

          public static int getRemoteParallelReduceTasks()

      Gets the total number of remote parallel reduce slots.
      Returns:
        number of remote parallel reduce tasks

    - #### setRemoteParallelReduceTasks

          public static void setRemoteParallelReduceTasks​(int preduce)

    - #### getLocalMaxMemory

          public static long getLocalMaxMemory()

      Gets the maximum memory [in bytes] of the current JVM.
      Returns:
        maximum memory of the current JVM

    - #### setLocalMaxMemory

          public static void setLocalMaxMemory​(long localMem)

    - #### getLocalMaxMemoryFraction

          public static double getLocalMaxMemoryFraction()

    - #### isLocalMode

          public static boolean isLocalMode()

    - #### isLocalMode

          public static boolean isLocalMode​(org.apache.hadoop.mapred.JobConf job)

    - #### getCkMaxCP

          public static int getCkMaxCP()

      Gets the maximum local parallelism constraint.
      Returns:
        maximum local parallelism constraint

    - #### getCkMaxMR

          public static int getCkMaxMR()

      Gets the maximum remote parallelism constraint
      Returns:
        maximum remote parallelism constraint

    - #### getCmMax

          public static long getCmMax()

      Gets the maximum memory constraint [in bytes].
      Returns:
        maximum memory constraint

    - #### getHDFSBlockSize

          public static long getHDFSBlockSize()

      Gets the HDFS blocksize of the used cluster in bytes.
      Returns:
        HDFS block size

    - #### extractMaxMemoryOpt

          public static long extractMaxMemoryOpt​(String javaOpts)

    - #### setMaxMemoryOpt

          public static void setMaxMemoryOpt​(org.apache.hadoop.mapred.JobConf job,
                                             String key,
                                             long bytes)

    - #### getClusterUtilization

          public static double getClusterUtilization​(boolean mapOnly)
                                              throws IOException

      Gets the fraction of running map/reduce tasks to existing map/reduce task slots. NOTE: on YARN the number of slots is a spurious indicator because containers are purely scheduled based on memory.
      Parameters:
        `mapOnly` - if true, only look at map tasks
      Returns:
        cluster utilization (current / capacity)
      Throws:
        `IOException` - if IOException occurs