org.apache.sysds.api.mlcontext

## Class MLContextUtil

- java.lang.Object
- - org.apache.sysds.api.mlcontext.MLContextUtil

-     public final class MLContextUtil
      extends Object

  Utility class containing methods for working with the MLContext API.

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static Class[]` | `ALL_SUPPORTED_DATA_TYPES` All data types supported by the MLContext API.
    `static Class[]` | `BASIC_DATA_TYPES` Basic data types supported by the MLContext API.
    `static String` | `BUILD_TIME_NOT_AVAILABLE` Build time not available message.
    `static Class[]` | `COMPLEX_DATA_TYPES` Complex data types supported by the MLContext API.
    `static String` | `VERSION_NOT_AVAILABLE` Version not available message.

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `MLContextUtil()`

  - ### Method Summary

    All Methods Static Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `static void` | `checkInputParameterType(String parameterName, Object parameterValue)` Verify that the type of input parameter value is supported.
    `static void` | `checkInputValueType(String name, Object value)` Verify that the type of input value is supported.
    `static void` | `checkInputValueTypes(Map<String,Object> inputs)` Verify that the types of input values are supported.
    `static Map<String,String>` | `convertInputParametersForParser(Map<String,Object> basicInputParameterMap)` Converts non-string basic input parameter values to strings to pass to the parser.
    `static Data` | `convertInputType(String parameterName, Object parameterValue)` Convert input types to internal SystemDS representations
    `static Data` | `convertInputType(String parameterName, Object parameterValue, Metadata metadata)` Convert input types to internal SystemDS representations
    `static void` | `deleteRemoveVariableInstructions(Program progam)` Delete the 'remove variable' instructions from a runtime program.
    `static String` | `determineOutputTypeAsString(LocalVariableMap symbolTable, String outputName)` Obtain a symbol table output type as a String
    `static String` | `displayInputs(String name, Map<String,Object> map, LocalVariableMap symbolTable)` Obtain a display of script inputs.
    `static String` | `displayMap(String mapName, Map<String,Object> map)` Display the keys and values in a Map
    `static String` | `displayOutputs(Set<String> outputNames, LocalVariableMap symbolTable)` Obtain a display of the script outputs.
    `static String` | `displayOutputs(String name, Set<String> outputNames, LocalVariableMap symbolTable)` Obtain a display of the script outputs.
    `static String` | `displaySet(String setName, Set<String> set)` Display the values in a Set
    `static String` | `displaySymbolTable(LocalVariableMap symbolTable)` Display the keys and values in the symbol table
    `static String` | `displaySymbolTable(String name, LocalVariableMap symbolTable)` Display the keys and values in the symbol table
    `static boolean` | `doesDataFrameLookLikeMatrix(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> df)` Examine the DataFrame schema to determine whether the data appears to be a matrix.
    `static boolean` | `doesSymbolTableContainFrameObject(LocalVariableMap symbolTable, String variableName)` Determine if the symbol table contains a FrameObject with the given variable name.
    `static boolean` | `doesSymbolTableContainMatrixObject(LocalVariableMap symbolTable, String variableName)` Determine if the symbol table contains a MatrixObject with the given variable name.
    `static String` | `getBasicTypeString(Object object)` Obtain the SystemDS scalar value type string equivalent of an accepted basic type (Integer, Boolean, Double, String)
    `static org.apache.spark.api.java.JavaSparkContext` | `getJavaSparkContext(MLContext mlContext)` Obtain the Java Spark Context
    `static org.apache.spark.api.java.JavaSparkContext` | `getJavaSparkContextFromProxy()` Obtain the Java Spark Context from the MLContextProxy
    `static org.apache.spark.SparkContext` | `getSparkContext(MLContext mlContext)` Obtain the Spark Context
    `static org.apache.spark.SparkContext` | `getSparkContextFromProxy()` Obtain the Spark Context from the MLContextProxy
    `static org.apache.spark.sql.SparkSession` | `getSparkSessionFromProxy()` Obtain the Spark Session from the MLContextProxy
    `static boolean` | `isBasicType(Object object)` Is the object one of the supported basic data types? (Integer, Boolean, Double, String)
    `static boolean` | `isComplexType(Object object)` Is the object one of the supported complex data types? (JavaRDD, RDD, DataFrame, Matrix, double[][], MatrixBlock, URL)
    `static boolean` | `isCSVLineAllNumbers(String line)` If no metadata is supplied for an RDD or JavaRDD, this method can be used to determine whether the data appears to be matrix (or a frame)
    `static boolean` | `isSparkVersionSupported(String sparkVersion, String minimumRecommendedSparkVersion)` Determine whether the Spark version is supported.
    `static String` | `quotedString(String str)` Return a double-quoted string with inner single and double quotes escaped.
    `static void` | `setCompilerConfig()` Set SystemDS compiler configuration properties for MLContext
    `static void` | `setConfig(String configFilePath)` Set SystemDS configuration properties based on a configuration file.
    `static void` | `setDefaultConfig()` Set default SystemDS configuration properties.
    `static void` | `verifySparkVersionSupported(org.apache.spark.sql.SparkSession spark)` Check that the Spark version is supported.
    `static String` | `welcomeMessage()` The SystemDS welcome message

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### VERSION_NOT_AVAILABLE

          public static final String VERSION_NOT_AVAILABLE

      Version not available message.
      See Also:
        Constant Field Values

    - #### BUILD_TIME_NOT_AVAILABLE

          public static final String BUILD_TIME_NOT_AVAILABLE

      Build time not available message.
      See Also:
        Constant Field Values

    - #### BASIC_DATA_TYPES

          public static final Class[] BASIC_DATA_TYPES

      Basic data types supported by the MLContext API.

    - #### COMPLEX_DATA_TYPES

          public static final Class[] COMPLEX_DATA_TYPES

      Complex data types supported by the MLContext API.

    - #### ALL_SUPPORTED_DATA_TYPES

          public static final Class[] ALL_SUPPORTED_DATA_TYPES

      All data types supported by the MLContext API.

  - ### Constructor Detail

    - #### MLContextUtil

          public MLContextUtil()

  - ### Method Detail

    - #### isSparkVersionSupported

          public static boolean isSparkVersionSupported(String sparkVersion,
                                                        String minimumRecommendedSparkVersion)

      Determine whether the Spark version is supported.
      Parameters:
        `sparkVersion` - Spark version string (ie, "1.5.0").
        `minimumRecommendedSparkVersion` - Minimum recommended Spark version string (ie, "2.1.0").
      Returns:
        `true` if Spark version supported; otherwise `false`.

    - #### verifySparkVersionSupported

          public static void verifySparkVersionSupported(org.apache.spark.sql.SparkSession spark)

      Check that the Spark version is supported. If it isn't supported, throw an MLContextException.
      Parameters:
        `spark` - SparkSession
      Throws:
        `MLContextException` - thrown if Spark version isn't supported

    - #### setDefaultConfig

          public static void setDefaultConfig()

      Set default SystemDS configuration properties.

    - #### setConfig

          public static void setConfig(String configFilePath)

      Set SystemDS configuration properties based on a configuration file.
      Parameters:
        `configFilePath` - Path to configuration file.
      Throws:
        `MLContextException` - if configuration file was not found or a parse exception occurred

    - #### setCompilerConfig

          public static void setCompilerConfig()

      Set SystemDS compiler configuration properties for MLContext

    - #### checkInputValueTypes

          public static void checkInputValueTypes(Map<String,Object> inputs)

      Verify that the types of input values are supported.
      Parameters:
        `inputs` - Map of String/Object pairs
      Throws:
        `MLContextException` - if an input value type is not supported

    - #### checkInputValueType

          public static void checkInputValueType(String name,
                                                 Object value)

      Verify that the type of input value is supported.
      Parameters:
        `name` - The name of the input
        `value` - The value of the input
      Throws:
        `MLContextException` - if the input value type is not supported

    - #### checkInputParameterType

          public static void checkInputParameterType(String parameterName,
                                                     Object parameterValue)

      Verify that the type of input parameter value is supported.
      Parameters:
        `parameterName` - The name of the input parameter
        `parameterValue` - The value of the input parameter
      Throws:
        `MLContextException` - if the input parameter value type is not supported

    - #### isBasicType

          public static boolean isBasicType(Object object)

      Is the object one of the supported basic data types? (Integer, Boolean, Double, String)
      Parameters:
        `object` - the object type to be examined
      Returns:
        `true` if type is a basic data type; otherwise `false`.

    - #### getBasicTypeString

          public static String getBasicTypeString(Object object)

      Obtain the SystemDS scalar value type string equivalent of an accepted basic type (Integer, Boolean, Double, String)
      Parameters:
        `object` - the object type to be examined
      Returns:
        a String representing the type as a SystemDS scalar value type

    - #### isComplexType

          public static boolean isComplexType(Object object)

      Is the object one of the supported complex data types? (JavaRDD, RDD, DataFrame, Matrix, double[][], MatrixBlock, URL)
      Parameters:
        `object` - the object type to be examined
      Returns:
        `true` if type is a complex data type; otherwise `false`.

    - #### convertInputParametersForParser

          public static Map<String,String> convertInputParametersForParser(Map<String,Object> basicInputParameterMap)

      Converts non-string basic input parameter values to strings to pass to the parser.
      Parameters:
        `basicInputParameterMap` - map of input parameters
      Returns:
        map of String/String name/value pairs

    - #### convertInputType

          public static Data convertInputType(String parameterName,
                                              Object parameterValue)

      Convert input types to internal SystemDS representations
      Parameters:
        `parameterName` - The name of the input parameter
        `parameterValue` - The value of the input parameter
      Returns:
        input in SystemDS data representation

    - #### convertInputType

          public static Data convertInputType(String parameterName,
                                              Object parameterValue,
                                              Metadata metadata)

      Convert input types to internal SystemDS representations
      Parameters:
        `parameterName` - The name of the input parameter
        `parameterValue` - The value of the input parameter
        `metadata` - matrix/frame metadata
      Returns:
        input in SystemDS data representation

    - #### isCSVLineAllNumbers

          public static boolean isCSVLineAllNumbers(String line)

      If no metadata is supplied for an RDD or JavaRDD, this method can be used to determine whether the data appears to be matrix (or a frame)
      Parameters:
        `line` - a line of the RDD
      Returns:
        `true` if all the csv-separated values are numbers, `false` otherwise

    - #### doesDataFrameLookLikeMatrix

          public static boolean doesDataFrameLookLikeMatrix(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> df)

      Examine the DataFrame schema to determine whether the data appears to be a matrix.
      Parameters:
        `df` - the DataFrame
      Returns:
        `true` if the DataFrame appears to be a matrix, `false` otherwise

    - #### quotedString

          public static String quotedString(String str)

      Return a double-quoted string with inner single and double quotes escaped.
      Parameters:
        `str` - the original string
      Returns:
        double-quoted string with inner single and double quotes escaped

    - #### displayMap

          public static String displayMap(String mapName,
                                          Map<String,Object> map)

      Display the keys and values in a Map
      Parameters:
        `mapName` - the name of the map
        `map` - Map of String keys and Object values
      Returns:
        the keys and values in the Map as a String

    - #### displaySet

          public static String displaySet(String setName,
                                          Set<String> set)

      Display the values in a Set
      Parameters:
        `setName` - the name of the Set
        `set` - Set of String values
      Returns:
        the values in the Set as a String

    - #### displaySymbolTable

          public static String displaySymbolTable(String name,
                                                  LocalVariableMap symbolTable)

      Display the keys and values in the symbol table
      Parameters:
        `name` - the name of the symbol table
        `symbolTable` - the LocalVariableMap
      Returns:
        the keys and values in the symbol table as a String

    - #### displaySymbolTable

          public static String displaySymbolTable(LocalVariableMap symbolTable)

      Display the keys and values in the symbol table
      Parameters:
        `symbolTable` - the LocalVariableMap
      Returns:
        the keys and values in the symbol table as a String

    - #### determineOutputTypeAsString

          public static String determineOutputTypeAsString(LocalVariableMap symbolTable,
                                                           String outputName)

      Obtain a symbol table output type as a String
      Parameters:
        `symbolTable` - the symbol table
        `outputName` - the name of the output variable
      Returns:
        the symbol table output type for a variable as a String

    - #### displayInputs

          public static String displayInputs(String name,
                                             Map<String,Object> map,
                                             LocalVariableMap symbolTable)

      Obtain a display of script inputs.
      Parameters:
        `name` - the title to display for the inputs
        `map` - the map of inputs
        `symbolTable` - the symbol table
      Returns:
        the script inputs represented as a String

    - #### displayOutputs

          public static String displayOutputs(String name,
                                              Set<String> outputNames,
                                              LocalVariableMap symbolTable)

      Obtain a display of the script outputs.
      Parameters:
        `name` - the title to display for the outputs
        `outputNames` - the names of the output variables
        `symbolTable` - the symbol table
      Returns:
        the script outputs represented as a String

    - #### displayOutputs

          public static String displayOutputs(Set<String> outputNames,
                                              LocalVariableMap symbolTable)

      Obtain a display of the script outputs.
      Parameters:
        `outputNames` - the names of the output variables
        `symbolTable` - the symbol table
      Returns:
        the script outputs represented as a String

    - #### welcomeMessage

          public static String welcomeMessage()

      The SystemDS welcome message
      Returns:
        the SystemDS welcome message

    - #### getSparkContext

          public static org.apache.spark.SparkContext getSparkContext(MLContext mlContext)

      Obtain the Spark Context
      Parameters:
        `mlContext` - the SystemDS MLContext
      Returns:
        the Spark Context

    - #### getJavaSparkContext

          public static org.apache.spark.api.java.JavaSparkContext getJavaSparkContext(MLContext mlContext)

      Obtain the Java Spark Context
      Parameters:
        `mlContext` - the SystemDS MLContext
      Returns:
        the Java Spark Context

    - #### getSparkContextFromProxy

          public static org.apache.spark.SparkContext getSparkContextFromProxy()

      Obtain the Spark Context from the MLContextProxy
      Returns:
        the Spark Context

    - #### getJavaSparkContextFromProxy

          public static org.apache.spark.api.java.JavaSparkContext getJavaSparkContextFromProxy()

      Obtain the Java Spark Context from the MLContextProxy
      Returns:
        the Java Spark Context

    - #### getSparkSessionFromProxy

          public static org.apache.spark.sql.SparkSession getSparkSessionFromProxy()

      Obtain the Spark Session from the MLContextProxy
      Returns:
        the Spark Session

    - #### doesSymbolTableContainFrameObject

          public static boolean doesSymbolTableContainFrameObject(LocalVariableMap symbolTable,
                                                                  String variableName)

      Determine if the symbol table contains a FrameObject with the given variable name.
      Parameters:
        `symbolTable` - the LocalVariableMap
        `variableName` - the variable name
      Returns:
        `true` if the variable in the symbol table is a FrameObject, `false` otherwise.

    - #### doesSymbolTableContainMatrixObject

          public static boolean doesSymbolTableContainMatrixObject(LocalVariableMap symbolTable,
                                                                   String variableName)

      Determine if the symbol table contains a MatrixObject with the given variable name.
      Parameters:
        `symbolTable` - the LocalVariableMap
        `variableName` - the variable name
      Returns:
        `true` if the variable in the symbol table is a MatrixObject, `false` otherwise.

    - #### deleteRemoveVariableInstructions

          public static void deleteRemoveVariableInstructions(Program progam)

      Delete the 'remove variable' instructions from a runtime program.
      Parameters:
        `progam` - runtime program

Copyright © 2021 The Apache Software Foundation. All rights reserved.