Package org.apache.sysds.api.mlcontext

## Class MLContextUtil

- java.lang.Object
- - org.apache.sysds.api.mlcontext.MLContextUtil

-     public final class MLContextUtil
      extends Object

  Utility class containing methods for working with the MLContext API.

- - ### Field Summary

    Fields
    Modifier and Type | Field | Description
    --- | --- | ---
    `static Class[]` | `ALL_SUPPORTED_DATA_TYPES` | All data types supported by the MLContext API.
    `static Class[]` | `BASIC_DATA_TYPES` | Basic data types supported by the MLContext API.
    `static String` | `BUILD_TIME_NOT_AVAILABLE` | Build time not available message.
    `static Class[]` | `COMPLEX_DATA_TYPES` | Complex data types supported by the MLContext API.
    `static String` | `VERSION_NOT_AVAILABLE` | Version not available message.

  - ### Constructor Summary

    Constructors
    Constructor | Description
    --- | ---
    `MLContextUtil()` |

  - ### Method Summary

    All Methods Static Methods Concrete Methods
    Modifier and Type | Method | Description
    --- | --- | ---
    `static void` | `checkInputParameterType​(String parameterName, Object parameterValue)` | Verify that the type of input parameter value is supported.
    `static void` | `checkInputValueType​(String name, Object value)` | Verify that the type of input value is supported.
    `static void` | `checkInputValueTypes​(Map<String,​Object> inputs)` | Verify that the types of input values are supported.
    `static Map<String,​String>` | `convertInputParametersForParser​(Map<String,​Object> basicInputParameterMap)` | Converts non-string basic input parameter values to strings to pass to the parser.
    `static Data` | `convertInputType​(String parameterName, Object parameterValue)` | Convert input types to internal SystemDS representations
    `static Data` | `convertInputType​(String parameterName, Object parameterValue, Metadata metadata)` | Convert input types to internal SystemDS representations
    `static void` | `deleteRemoveVariableInstructions​(Program progam)` | Delete the 'remove variable' instructions from a runtime program.
    `static String` | `determineOutputTypeAsString​(LocalVariableMap symbolTable, String outputName)` | Obtain a symbol table output type as a String
    `static String` | `displayInputs​(String name, Map<String,​Object> map, LocalVariableMap symbolTable)` | Obtain a display of script inputs.
    `static String` | `displayMap​(String mapName, Map<String,​Object> map)` | Display the keys and values in a Map
    `static String` | `displayOutputs​(String name, Set<String> outputNames, LocalVariableMap symbolTable)` | Obtain a display of the script outputs.
    `static String` | `displayOutputs​(Set<String> outputNames, LocalVariableMap symbolTable)` | Obtain a display of the script outputs.
    `static String` | `displaySet​(String setName, Set<String> set)` | Display the values in a Set
    `static String` | `displaySymbolTable​(String name, LocalVariableMap symbolTable)` | Display the keys and values in the symbol table
    `static String` | `displaySymbolTable​(LocalVariableMap symbolTable)` | Display the keys and values in the symbol table
    `static boolean` | `doesDataFrameLookLikeMatrix​(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> df)` | Examine the DataFrame schema to determine whether the data appears to be a matrix.
    `static boolean` | `doesSymbolTableContainFrameObject​(LocalVariableMap symbolTable, String variableName)` | Determine if the symbol table contains a FrameObject with the given variable name.
    `static boolean` | `doesSymbolTableContainMatrixObject​(LocalVariableMap symbolTable, String variableName)` | Determine if the symbol table contains a MatrixObject with the given variable name.
    `static String` | `getBasicTypeString​(Object object)` | Obtain the SystemDS scalar value type string equivalent of an accepted basic type (Integer, Boolean, Double, String)
    `static org.apache.spark.api.java.JavaSparkContext` | `getJavaSparkContext​(MLContext mlContext)` | Obtain the Java Spark Context
    `static org.apache.spark.api.java.JavaSparkContext` | `getJavaSparkContextFromProxy()` | Obtain the Java Spark Context from the MLContextProxy
    `static org.apache.spark.SparkContext` | `getSparkContext​(MLContext mlContext)` | Obtain the Spark Context
    `static org.apache.spark.SparkContext` | `getSparkContextFromProxy()` | Obtain the Spark Context from the MLContextProxy
    `static org.apache.spark.sql.SparkSession` | `getSparkSessionFromProxy()` | Obtain the Spark Session from the MLContextProxy
    `static boolean` | `isBasicType​(Object object)` | Is the object one of the supported basic data types? (Integer, Boolean, Double, String)
    `static boolean` | `isComplexType​(Object object)` | Is the object one of the supported complex data types? (JavaRDD, RDD, DataFrame, Matrix, double[][], MatrixBlock, URL)
    `static boolean` | `isCSVLineAllNumbers​(String line)` | If no metadata is supplied for an RDD or JavaRDD, this method can be used to determine whether the data appears to be matrix (or a frame)
    `static boolean` | `isSparkVersionSupported​(String sparkVersion, String minimumRecommendedSparkVersion)` | Determine whether the Spark version is supported.
    `static String` | `quotedString​(String str)` | Return a double-quoted string with inner single and double quotes escaped.
    `static void` | `setCompilerConfig()` | Set SystemDS compiler configuration properties for MLContext
    `static void` | `setConfig​(String configFilePath)` | Set SystemDS configuration properties based on a configuration file.
    `static void` | `setDefaultConfig()` | Set default SystemDS configuration properties.
    `static void` | `verifySparkVersionSupported​(org.apache.spark.sql.SparkSession spark)` | Check that the Spark version is supported.
    `static String` | `welcomeMessage()` | The SystemDS welcome message

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### VERSION_NOT_AVAILABLE

          public static final String VERSION_NOT_AVAILABLE

      Version not available message.
      See Also:
        Constant Field Values

    - #### BUILD_TIME_NOT_AVAILABLE

          public static final String BUILD_TIME_NOT_AVAILABLE

      Build time not available message.
      See Also:
        Constant Field Values

    - #### BASIC_DATA_TYPES

          public static final Class[] BASIC_DATA_TYPES

      Basic data types supported by the MLContext API.

    - #### COMPLEX_DATA_TYPES

          public static final Class[] COMPLEX_DATA_TYPES

      Complex data types supported by the MLContext API.

    - #### ALL_SUPPORTED_DATA_TYPES

          public static final Class[] ALL_SUPPORTED_DATA_TYPES

      All data types supported by the MLContext API.

  - ### Constructor Detail

    - #### MLContextUtil

          public MLContextUtil()

  - ### Method Detail

    - #### isSparkVersionSupported

          public static boolean isSparkVersionSupported​(String sparkVersion,
                                                        String minimumRecommendedSparkVersion)

      Determine whether the Spark version is supported.
      Parameters:
        `sparkVersion` - Spark version string (ie, "1.5.0").
        `minimumRecommendedSparkVersion` - Minimum recommended Spark version string (ie, "2.1.0").
      Returns:
        `true` if Spark version supported; otherwise `false`.

    - #### verifySparkVersionSupported

          public static void verifySparkVersionSupported​(org.apache.spark.sql.SparkSession spark)

      Check that the Spark version is supported. If it isn't supported, throw an MLContextException.
      Parameters:
        `spark` - SparkSession
      Throws:
        `MLContextException` - thrown if Spark version isn't supported

    - #### setDefaultConfig

          public static void setDefaultConfig()

      Set default SystemDS configuration properties.

    - #### setConfig

          public static void setConfig​(String configFilePath)

      Set SystemDS configuration properties based on a configuration file.
      Parameters:
        `configFilePath` - Path to configuration file.
      Throws:
        `MLContextException` - if configuration file was not found or a parse exception occurred

    - #### setCompilerConfig

          public static void setCompilerConfig()

      Set SystemDS compiler configuration properties for MLContext

    - #### checkInputValueTypes

          public static void checkInputValueTypes​(Map<String,​Object> inputs)

      Verify that the types of input values are supported.
      Parameters:
        `inputs` - Map of String/Object pairs
      Throws:
        `MLContextException` - if an input value type is not supported

    - #### checkInputValueType

          public static void checkInputValueType​(String name,
                                                 Object value)

      Verify that the type of input value is supported.
      Parameters:
        `name` - The name of the input
        `value` - The value of the input
      Throws:
        `MLContextException` - if the input value type is not supported

    - #### checkInputParameterType

          public static void checkInputParameterType​(String parameterName,
                                                     Object parameterValue)

      Verify that the type of input parameter value is supported.
      Parameters:
        `parameterName` - The name of the input parameter
        `parameterValue` - The value of the input parameter
      Throws:
        `MLContextException` - if the input parameter value type is not supported

    - #### isBasicType

          public static boolean isBasicType​(Object object)

      Is the object one of the supported basic data types? (Integer, Boolean, Double, String)
      Parameters:
        `object` - the object type to be examined
      Returns:
        `true` if type is a basic data type; otherwise `false`.

    - #### getBasicTypeString

          public static String getBasicTypeString​(Object object)

      Obtain the SystemDS scalar value type string equivalent of an accepted basic type (Integer, Boolean, Double, String)
      Parameters:
        `object` - the object type to be examined
      Returns:
        a String representing the type as a SystemDS scalar value type

    - #### isComplexType

          public static boolean isComplexType​(Object object)

      Is the object one of the supported complex data types? (JavaRDD, RDD, DataFrame, Matrix, double[][], MatrixBlock, URL)
      Parameters:
        `object` - the object type to be examined
      Returns:
        `true` if type is a complex data type; otherwise `false`.

    - #### convertInputParametersForParser

          public static Map<String,​String> convertInputParametersForParser​(Map<String,​Object> basicInputParameterMap)

      Converts non-string basic input parameter values to strings to pass to the parser.
      Parameters:
        `basicInputParameterMap` - map of input parameters
      Returns:
        map of String/String name/value pairs

    - #### convertInputType

          public static Data convertInputType​(String parameterName,
                                              Object parameterValue)

      Convert input types to internal SystemDS representations
      Parameters:
        `parameterName` - The name of the input parameter
        `parameterValue` - The value of the input parameter
      Returns:
        input in SystemDS data representation

    - #### convertInputType

          public static Data convertInputType​(String parameterName,
                                              Object parameterValue,
                                              Metadata metadata)

      Convert input types to internal SystemDS representations
      Parameters:
        `parameterName` - The name of the input parameter
        `parameterValue` - The value of the input parameter
        `metadata` - matrix/frame metadata
      Returns:
        input in SystemDS data representation

    - #### isCSVLineAllNumbers

          public static boolean isCSVLineAllNumbers​(String line)

      If no metadata is supplied for an RDD or JavaRDD, this method can be used to determine whether the data appears to be matrix (or a frame)
      Parameters:
        `line` - a line of the RDD
      Returns:
        `true` if all the csv-separated values are numbers, `false` otherwise

    - #### doesDataFrameLookLikeMatrix

          public static boolean doesDataFrameLookLikeMatrix​(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> df)

      Examine the DataFrame schema to determine whether the data appears to be a matrix.
      Parameters:
        `df` - the DataFrame
      Returns:
        `true` if the DataFrame appears to be a matrix, `false` otherwise

    - #### quotedString

          public static String quotedString​(String str)

      Return a double-quoted string with inner single and double quotes escaped.
      Parameters:
        `str` - the original string
      Returns:
        double-quoted string with inner single and double quotes escaped

    - #### displayMap

          public static String displayMap​(String mapName,
                                          Map<String,​Object> map)

      Display the keys and values in a Map
      Parameters:
        `mapName` - the name of the map
        `map` - Map of String keys and Object values
      Returns:
        the keys and values in the Map as a String

    - #### displaySet

          public static String displaySet​(String setName,
                                          Set<String> set)

      Display the values in a Set
      Parameters:
        `setName` - the name of the Set
        `set` - Set of String values
      Returns:
        the values in the Set as a String

    - #### displaySymbolTable

          public static String displaySymbolTable​(String name,
                                                  LocalVariableMap symbolTable)

      Display the keys and values in the symbol table
      Parameters:
        `name` - the name of the symbol table
        `symbolTable` - the LocalVariableMap
      Returns:
        the keys and values in the symbol table as a String

    - #### displaySymbolTable

          public static String displaySymbolTable​(LocalVariableMap symbolTable)

      Display the keys and values in the symbol table
      Parameters:
        `symbolTable` - the LocalVariableMap
      Returns:
        the keys and values in the symbol table as a String

    - #### determineOutputTypeAsString

          public static String determineOutputTypeAsString​(LocalVariableMap symbolTable,
                                                           String outputName)

      Obtain a symbol table output type as a String
      Parameters:
        `symbolTable` - the symbol table
        `outputName` - the name of the output variable
      Returns:
        the symbol table output type for a variable as a String

    - #### displayInputs

          public static String displayInputs​(String name,
                                             Map<String,​Object> map,
                                             LocalVariableMap symbolTable)

      Obtain a display of script inputs.
      Parameters:
        `name` - the title to display for the inputs
        `map` - the map of inputs
        `symbolTable` - the symbol table
      Returns:
        the script inputs represented as a String

    - #### displayOutputs

          public static String displayOutputs​(String name,
                                              Set<String> outputNames,
                                              LocalVariableMap symbolTable)

      Obtain a display of the script outputs.
      Parameters:
        `name` - the title to display for the outputs
        `outputNames` - the names of the output variables
        `symbolTable` - the symbol table
      Returns:
        the script outputs represented as a String

    - #### displayOutputs

          public static String displayOutputs​(Set<String> outputNames,
                                              LocalVariableMap symbolTable)

      Obtain a display of the script outputs.
      Parameters:
        `outputNames` - the names of the output variables
        `symbolTable` - the symbol table
      Returns:
        the script outputs represented as a String

    - #### welcomeMessage

          public static String welcomeMessage()

      The SystemDS welcome message
      Returns:
        the SystemDS welcome message

    - #### getSparkContext

          public static org.apache.spark.SparkContext getSparkContext​(MLContext mlContext)

      Obtain the Spark Context
      Parameters:
        `mlContext` - the SystemDS MLContext
      Returns:
        the Spark Context

    - #### getJavaSparkContext

          public static org.apache.spark.api.java.JavaSparkContext getJavaSparkContext​(MLContext mlContext)

      Obtain the Java Spark Context
      Parameters:
        `mlContext` - the SystemDS MLContext
      Returns:
        the Java Spark Context

    - #### getSparkContextFromProxy

          public static org.apache.spark.SparkContext getSparkContextFromProxy()

      Obtain the Spark Context from the MLContextProxy
      Returns:
        the Spark Context

    - #### getJavaSparkContextFromProxy

          public static org.apache.spark.api.java.JavaSparkContext getJavaSparkContextFromProxy()

      Obtain the Java Spark Context from the MLContextProxy
      Returns:
        the Java Spark Context

    - #### getSparkSessionFromProxy

          public static org.apache.spark.sql.SparkSession getSparkSessionFromProxy()

      Obtain the Spark Session from the MLContextProxy
      Returns:
        the Spark Session

    - #### doesSymbolTableContainFrameObject

          public static boolean doesSymbolTableContainFrameObject​(LocalVariableMap symbolTable,
                                                                  String variableName)

      Determine if the symbol table contains a FrameObject with the given variable name.
      Parameters:
        `symbolTable` - the LocalVariableMap
        `variableName` - the variable name
      Returns:
        `true` if the variable in the symbol table is a FrameObject, `false` otherwise.

    - #### doesSymbolTableContainMatrixObject

          public static boolean doesSymbolTableContainMatrixObject​(LocalVariableMap symbolTable,
                                                                   String variableName)

      Determine if the symbol table contains a MatrixObject with the given variable name.
      Parameters:
        `symbolTable` - the LocalVariableMap
        `variableName` - the variable name
      Returns:
        `true` if the variable in the symbol table is a MatrixObject, `false` otherwise.

    - #### deleteRemoveVariableInstructions

          public static void deleteRemoveVariableInstructions​(Program progam)

      Delete the 'remove variable' instructions from a runtime program.
      Parameters:
        `progam` - runtime program