org.apache.sysds.api.mlcontext

## Class MLContext

- java.lang.Object
- - org.apache.sysds.api.mlcontext.MLContext

- All Implemented Interfaces:
    ConfigurableAPI

      public class MLContext
      extends Object
      implements ConfigurableAPI

  The MLContext API offers programmatic access to SystemDS on Spark from languages such as Scala, Java, and Python.

- - ### Nested Class Summary

    Nested Classes
    Modifier and Type | Class and Description
    --- | ---
    `static class` | `MLContext.ExecutionType` The different types of execution environments supported by SystemDS.
    `static class` | `MLContext.ExplainLevel` The different explain levels supported by SystemDS.
    `class` | `MLContext.InternalProxy` Used internally by MLContextProxy.

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `MLContext(org.apache.spark.api.java.JavaSparkContext javaSparkContext)` Create an MLContext based on a JavaSparkContext for interaction with SystemDS on Spark.
    `MLContext(org.apache.spark.SparkContext sparkContext)` Create an MLContext based on a SparkContext for interaction with SystemDS on Spark.
    `MLContext(org.apache.spark.sql.SparkSession spark)` Create an MLContext based on a SparkSession for interaction with SystemDS on Spark.

  - ### Method Summary

    All Methods Static Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `String` | `buildTime()` Obtain the SystemDS jar file build time.
    `void` | `close()` Closes the mlcontext, which includes the cleanup of static and local state as well as scratch space and buffer pool cleanup.
    `MLResults` | `execute(Script script)` Execute a DML or PYDML Script.
    `MLResults` | `execute(Script script, ScriptExecutor scriptExecutor)` Execute a DML or PYDML Script object using a ScriptExecutor.
    `static MLContext` | `getActiveMLContext()` Retrieve the currently active MLContext.
    `MLContext.ExecutionType` | `getExecutionType()` Obtain the current execution environment.
    `MLContext.InternalProxy` | `getInternalProxy()` Used internally by MLContextProxy.
    `org.apache.spark.sql.SparkSession` | `getSparkSession()` Obtain the SparkSession associated with this MLContext.
    `int` | `getStatisticsMaxHeavyHitters()` Obtain the maximum number of heavy hitters that are printed out as part of the statistics.
    `ProjectInfo` | `info()` Obtain information about the project such as version and build time from the manifest in the SystemDS jar file.
    `boolean` | `isExplain()` Whether or not an explanation of the DML/PYDML program should be output to standard output.
    `boolean` | `isForceGPU()` Whether or not the "force" GPU mode is enabled.
    `boolean` | `isGPU()` Whether or not the GPU mode is enabled.
    `boolean` | `isInitBeforeExecution()` Whether or not the default ScriptExecutor should be initialized before execution.
    `boolean` | `isMaintainSymbolTable()` Obtain whether or not all values should be maintained in the symbol table after execution.
    `boolean` | `isStatistics()` Whether or not statistics of the DML/PYDML program execution should be output to standard output.
    `void` | `resetConfig()` Reset configuration settings to default settings.
    `void` | `setConfig(String configFilePath)` Set SystemDS configuration based on a configuration file.
    `void` | `setConfigProperty(String propertyName, String propertyValue)` Set configuration property, such as `setConfigProperty("sysds.localtmpdir", "/tmp/systemds")`.
    `void` | `setExecutionScript(Script executionScript)` Sets the script that is being executed
    `void` | `setExecutionType(MLContext.ExecutionType executionType)` Set the execution environment.
    `void` | `setExplain(boolean explain)` Whether or not an explanation of the DML/PYDML program should be output to standard output.
    `void` | `setExplainLevel(MLContext.ExplainLevel explainLevel)` Set the level of program explanation that should be displayed if explain is set to true.
    `void` | `setExplainLevel(String explainLevel)` Set the level of program explanation that should be displayed if explain is set to true.
    `void` | `setForceGPU(boolean enable)` Whether or not to explicitly "force" the usage of GPU.
    `void` | `setGPU(boolean enable)` Whether or not to use (an available) GPU on the driver node.
    `void` | `setInitBeforeExecution(boolean initBeforeExecution)` Whether or not the default ScriptExecutor should be initialized before execution.
    `void` | `setLineage(boolean lineage)` Set whether or not lineage should be traced
    `void` | `setLineage(LineageCacheConfig.ReuseCacheType reuse)` Set type of lineage-based reuse caching and enable lineage tracing
    `void` | `setMaintainSymbolTable(boolean maintainSymbolTable)` Set whether or not all values should be maintained in the symbol table after execution.
    `void` | `setStatistics(boolean statistics)` Whether or not statistics of the DML/PYDML program execution should be output to standard output.
    `void` | `setStatisticsMaxHeavyHitters(int maxHeavyHitters)` Sets the maximum number of heavy hitters that are printed out as part of the statistics.
    `String` | `version()` Obtain the SystemDS version number.

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Constructor Detail

    - #### MLContext

          public MLContext(org.apache.spark.sql.SparkSession spark)

      Create an MLContext based on a SparkSession for interaction with SystemDS on Spark.
      Parameters:
        `spark` - SparkSession

    - #### MLContext

          public MLContext(org.apache.spark.SparkContext sparkContext)

      Create an MLContext based on a SparkContext for interaction with SystemDS on Spark.
      Parameters:
        `sparkContext` - SparkContext

    - #### MLContext

          public MLContext(org.apache.spark.api.java.JavaSparkContext javaSparkContext)

      Create an MLContext based on a JavaSparkContext for interaction with SystemDS on Spark.
      Parameters:
        `javaSparkContext` - JavaSparkContext

  - ### Method Detail

    - #### getActiveMLContext

          public static MLContext getActiveMLContext()

      Retrieve the currently active MLContext. This is used internally by SystemDS via MLContextProxy.
      Returns:
        the active MLContext

    - #### resetConfig

          public void resetConfig()

      Description copied from interface: `ConfigurableAPI`
      Reset configuration settings to default settings.
      Specified by:
        `resetConfig` in interface `ConfigurableAPI`

    - #### setConfigProperty

          public void setConfigProperty(String propertyName,
                                        String propertyValue)

      Description copied from interface: `ConfigurableAPI`
      Set configuration property, such as `setConfigProperty("sysds.localtmpdir", "/tmp/systemds")`.
      Specified by:
        `setConfigProperty` in interface `ConfigurableAPI`
      Parameters:
        `propertyName` - property name
        `propertyValue` - property value

    - #### execute

          public MLResults execute(Script script)

      Execute a DML or PYDML Script.
      Parameters:
        `script` - The DML or PYDML Script object to execute.
      Returns:
        the results as a MLResults object

    - #### execute

          public MLResults execute(Script script,
                                   ScriptExecutor scriptExecutor)

      Execute a DML or PYDML Script object using a ScriptExecutor. The ScriptExecutor class can be extended to allow the modification of the default execution pathway.
      Parameters:
        `script` - the DML or PYDML Script object
        `scriptExecutor` - the ScriptExecutor that defines the script execution pathway
      Returns:
        the results as a MLResults object

    - #### setExecutionScript

          public void setExecutionScript(Script executionScript)

      Sets the script that is being executed
      Parameters:
        `executionScript` - script that is being executed

    - #### setConfig

          public void setConfig(String configFilePath)

      Set SystemDS configuration based on a configuration file.
      Parameters:
        `configFilePath` - path to the configuration file

    - #### getSparkSession

          public org.apache.spark.sql.SparkSession getSparkSession()

      Obtain the SparkSession associated with this MLContext.
      Returns:
        the SparkSession associated with this MLContext.

    - #### isExplain

          public boolean isExplain()

      Whether or not an explanation of the DML/PYDML program should be output to standard output.
      Returns:
        `true` if explanation should be output, `false` otherwise

    - #### setExplain

          public void setExplain(boolean explain)

      Whether or not an explanation of the DML/PYDML program should be output to standard output.
      Parameters:
        `explain` - `true` if explanation should be output, `false` otherwise

    - #### setLineage

          public void setLineage(boolean lineage)

      Set whether or not lineage should be traced
      Parameters:
        `lineage` - `true` if lineage should be traced, `false` otherwise

    - #### setLineage

          public void setLineage(LineageCacheConfig.ReuseCacheType reuse)

      Set type of lineage-based reuse caching and enable lineage tracing
      Parameters:
        `reuse` - reuse cache type to use

    - #### isMaintainSymbolTable

          public boolean isMaintainSymbolTable()

      Obtain whether or not all values should be maintained in the symbol table after execution.
      Returns:
        `true` if all values should be maintained in the symbol table, `false` otherwise

    - #### setMaintainSymbolTable

          public void setMaintainSymbolTable(boolean maintainSymbolTable)

      Set whether or not all values should be maintained in the symbol table after execution.
      Parameters:
        `maintainSymbolTable` - `true` if all values should be maintained in the symbol table, `false` otherwise

    - #### setExplainLevel

          public void setExplainLevel(MLContext.ExplainLevel explainLevel)

      Set the level of program explanation that should be displayed if explain is set to true.
      Parameters:
        `explainLevel` - the level of program explanation

    - #### setExplainLevel

          public void setExplainLevel(String explainLevel)

      Set the level of program explanation that should be displayed if explain is set to true.
      Parameters:
        `explainLevel` - string denoting program explanation

    - #### setGPU

          public void setGPU(boolean enable)

      Whether or not to use (an available) GPU on the driver node. If a GPU is not available, and the GPU mode is set, SystemDS will crash when the program is run.
      Parameters:
        `enable` - true if needs to be enabled, false otherwise

    - #### setForceGPU

          public void setForceGPU(boolean enable)

      Whether or not to explicitly "force" the usage of GPU. If a GPU is not available, and the GPU mode is set or if available memory on GPU is less, SystemDS will crash when the program is run.
      Parameters:
        `enable` - true if needs to be enabled, false otherwise

    - #### isGPU

          public boolean isGPU()

      Whether or not the GPU mode is enabled.
      Returns:
        true if enabled, false otherwise

    - #### isForceGPU

          public boolean isForceGPU()

      Whether or not the "force" GPU mode is enabled.
      Returns:
        true if enabled, false otherwise

    - #### getInternalProxy

          public MLContext.InternalProxy getInternalProxy()

      Used internally by MLContextProxy.
      Returns:
        InternalProxy object used by MLContextProxy

    - #### isStatistics

          public boolean isStatistics()

      Whether or not statistics of the DML/PYDML program execution should be output to standard output.
      Returns:
        `true` if statistics should be output, `false` otherwise

    - #### setStatistics

          public void setStatistics(boolean statistics)

      Whether or not statistics of the DML/PYDML program execution should be output to standard output.
      Parameters:
        `statistics` - `true` if statistics should be output, `false` otherwise

    - #### setStatisticsMaxHeavyHitters

          public void setStatisticsMaxHeavyHitters(int maxHeavyHitters)

      Sets the maximum number of heavy hitters that are printed out as part of the statistics.
      Parameters:
        `maxHeavyHitters` - maximum number of heavy hitters to print

    - #### close

          public void close()

      Closes the mlcontext, which includes the cleanup of static and local state as well as scratch space and buffer pool cleanup. Note that the spark context is not explicitly closed to allow external reuse.

    - #### info

          public ProjectInfo info()

      Obtain information about the project such as version and build time from the manifest in the SystemDS jar file.
      Returns:
        information about the project

    - #### version

          public String version()

      Obtain the SystemDS version number.
      Returns:
        the SystemDS version number

    - #### buildTime

          public String buildTime()

      Obtain the SystemDS jar file build time.
      Returns:
        the SystemDS jar file build time

    - #### getStatisticsMaxHeavyHitters

          public int getStatisticsMaxHeavyHitters()

      Obtain the maximum number of heavy hitters that are printed out as part of the statistics.
      Returns:
        maximum number of heavy hitters to print

    - #### isInitBeforeExecution

          public boolean isInitBeforeExecution()

      Whether or not the default ScriptExecutor should be initialized before execution.
      Returns:
        `true` if ScriptExecutor should be initialized before execution, `false` otherwise

    - #### setInitBeforeExecution

          public void setInitBeforeExecution(boolean initBeforeExecution)

      Whether or not the default ScriptExecutor should be initialized before execution.
      Parameters:
        `initBeforeExecution` - `true` if ScriptExecutor should be initialized before execution, `false` otherwise

    - #### getExecutionType

          public MLContext.ExecutionType getExecutionType()

      Obtain the current execution environment.
      Returns:
        the execution environment

    - #### setExecutionType

          public void setExecutionType(MLContext.ExecutionType executionType)

      Set the execution environment.
      Parameters:
        `executionType` - the execution environment

Copyright © 2021 The Apache Software Foundation. All rights reserved.