org.apache.sysds.runtime.controlprogram.parfor.stat

## Class InfrastructureAnalyzer

- java.lang.Object
- - org.apache.sysds.runtime.controlprogram.parfor.stat.InfrastructureAnalyzer

-     public class InfrastructureAnalyzer
      extends Object

  Central place for analyzing and obtaining static infrastructure properties such as memory and number of logical processors.

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static long` | `DEFAULT_JVM_SIZE`
    `static String` | `MR_FRAMEWORK_NAME`
    `static String` | `MR_JOBTRACKER_ADDRESS`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `InfrastructureAnalyzer()`

  - ### Method Summary

    All Methods Static Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `static long` | `extractMaxMemoryOpt(String javaOpts)`
    `static int` | `getCkMaxCP()` Gets the maximum local parallelism constraint.
    `static int` | `getCkMaxMR()` Gets the maximum remote parallelism constraint
    `static double` | `getClusterUtilization(boolean mapOnly)` Gets the fraction of running map/reduce tasks to existing map/reduce task slots.
    `static long` | `getCmMax()` Gets the maximum memory constraint [in bytes].
    `static long` | `getHDFSBlockSize()` Gets the HDFS blocksize of the used cluster in bytes.
    `static long` | `getLocalMaxMemory()` Gets the maximum memory [in bytes] of the current JVM.
    `static double` | `getLocalMaxMemoryFraction()`
    `static int` | `getLocalParallelism()` Gets the number of logical processors of the current node, including hyper-threading if enabled.
    `static int` | `getRemoteParallelMapTasks()` Gets the number of remote parallel map slots.
    `static int` | `getRemoteParallelNodes()` Gets the number of cluster nodes (number of tasktrackers).
    `static int` | `getRemoteParallelReduceTasks()` Gets the total number of remote parallel reduce slots.
    `static boolean` | `isLocalMode()`
    `static boolean` | `isLocalMode(org.apache.hadoop.mapred.JobConf job)`
    `static void` | `setLocalMaxMemory(long localMem)`
    `static void` | `setMaxMemoryOpt(org.apache.hadoop.mapred.JobConf job, String key, long bytes)`
    `static void` | `setRemoteParallelMapTasks(int pmap)`
    `static void` | `setRemoteParallelReduceTasks(int preduce)`

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### DEFAULT_JVM_SIZE

          public static final long DEFAULT_JVM_SIZE

      See Also:
        Constant Field Values

    - #### MR_FRAMEWORK_NAME

          public static final String MR_FRAMEWORK_NAME

      See Also:
        Constant Field Values

    - #### MR_JOBTRACKER_ADDRESS

          public static final String MR_JOBTRACKER_ADDRESS

      See Also:
        Constant Field Values

  - ### Constructor Detail

    - #### InfrastructureAnalyzer

          public InfrastructureAnalyzer()

  - ### Method Detail

    - #### getLocalParallelism

          public static int getLocalParallelism()

      Gets the number of logical processors of the current node, including hyper-threading if enabled.
      Returns:
        number of local processors of the current node

    - #### getRemoteParallelNodes

          public static int getRemoteParallelNodes()

      Gets the number of cluster nodes (number of tasktrackers). If multiple tasktracker are started per node, each tasktracker is viewed as individual node.
      Returns:
        number of cluster nodes

    - #### getRemoteParallelMapTasks

          public static int getRemoteParallelMapTasks()

      Gets the number of remote parallel map slots.
      Returns:
        number of remote parallel map tasks

    - #### setRemoteParallelMapTasks

          public static void setRemoteParallelMapTasks(int pmap)

    - #### getRemoteParallelReduceTasks

          public static int getRemoteParallelReduceTasks()

      Gets the total number of remote parallel reduce slots.
      Returns:
        number of remote parallel reduce tasks

    - #### setRemoteParallelReduceTasks

          public static void setRemoteParallelReduceTasks(int preduce)

    - #### getLocalMaxMemory

          public static long getLocalMaxMemory()

      Gets the maximum memory [in bytes] of the current JVM.
      Returns:
        maximum memory of the current JVM

    - #### setLocalMaxMemory

          public static void setLocalMaxMemory(long localMem)

    - #### getLocalMaxMemoryFraction

          public static double getLocalMaxMemoryFraction()

    - #### isLocalMode

          public static boolean isLocalMode()

    - #### isLocalMode

          public static boolean isLocalMode(org.apache.hadoop.mapred.JobConf job)

    - #### getCkMaxCP

          public static int getCkMaxCP()

      Gets the maximum local parallelism constraint.
      Returns:
        maximum local parallelism constraint

    - #### getCkMaxMR

          public static int getCkMaxMR()

      Gets the maximum remote parallelism constraint
      Returns:
        maximum remote parallelism constraint

    - #### getCmMax

          public static long getCmMax()

      Gets the maximum memory constraint [in bytes].
      Returns:
        maximum memory constraint

    - #### getHDFSBlockSize

          public static long getHDFSBlockSize()

      Gets the HDFS blocksize of the used cluster in bytes.
      Returns:
        HDFS block size

    - #### extractMaxMemoryOpt

          public static long extractMaxMemoryOpt(String javaOpts)

    - #### setMaxMemoryOpt

          public static void setMaxMemoryOpt(org.apache.hadoop.mapred.JobConf job,
                                             String key,
                                             long bytes)

    - #### getClusterUtilization

          public static double getClusterUtilization(boolean mapOnly)
                                              throws IOException

      Gets the fraction of running map/reduce tasks to existing map/reduce task slots. NOTE: on YARN the number of slots is a spurious indicator because containers are purely scheduled based on memory.
      Parameters:
        `mapOnly` - if true, only look at map tasks
      Returns:
        cluster utilization (current / capacity)
      Throws:
        `IOException` - if IOException occurs

Copyright © 2021 The Apache Software Foundation. All rights reserved.