org.apache.sysds.api.jmlc

## Class Connection

- java.lang.Object
- - org.apache.sysds.api.jmlc.Connection

- All Implemented Interfaces:
    Closeable, AutoCloseable

      public class Connection
      extends Object
      implements Closeable

  Interaction with SystemDS using the JMLC (Java Machine Learning Connector) API is initiated with a `Connection` object. The JMLC API is designed after JDBC. A DML script is precompiled by calling the `prepareScript(String, String[], String[])` method or the `prepareScript(String, Map, String[], String[])` method on the `Connection` object, which returns a `PreparedScript` object. Note that this is similar to calling a `prepareStatement` method on a JDBC `Connection` object.

  Following this, input variable data is passed to the script by calling the `setFrame`, `setMatrix`, and `setScalar` methods of the `PreparedScript` object. The script is executed via `PreparedScript`'s `executeScript` method, which returns a `ResultVariables` object, which is similar to a JDBC `ResultSet`. Data can be read from a `ResultVariables` object by calling its `getFrame` and `getMatrix` methods.

- - ### Constructor Summary

    Constructors
    Constructor and Description
    `Connection()` Connection constructor, the starting point for any other JMLC API calls.
    `Connection(CompilerConfig.ConfigType... cconfigs)` Connection constructor, the starting point for any other JMLC API calls.
    `Connection(DMLConfig dmlconfig)` Connection constructor, the starting point for any other JMLC API calls.
    `Connection(DMLConfig dmlconfig, CompilerConfig.ConfigType... cconfigs)` Connection constructor, the starting point for any other JMLC API calls.

  - ### Method Summary

    All Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `void` | `close()` Close connection to SystemDS, which clears the thread-local DML and compiler configurations.
    `double[][]` | `convertToDoubleMatrix(InputStream input, int rows, int cols)` Converts an input stream of a string matrix in textcell format into a dense double array.
    `double[][]` | `convertToDoubleMatrix(InputStream input, int rows, int cols, String format)` Converts an input stream of a string matrix in csv or textcell format into a dense double array.
    `double[][]` | `convertToDoubleMatrix(String input, int rows, int cols)` Converts an input string representation of a matrix in textcell format into a dense double array.
    `double[][]` | `convertToDoubleMatrix(String input, String meta)` Converts an input string representation of a matrix in csv or textcell format into a dense double array.
    `FrameBlock` | `convertToFrame(InputStream input, int rows, int cols)` Converts an input stream of a string frame in textcell format into a frame block.
    `FrameBlock` | `convertToFrame(InputStream input, int rows, int cols, String format)` Converts an input stream of a frame in csv or textcell format into a frame block.
    `FrameBlock` | `convertToFrame(InputStream input, String meta)` Converts an input stream of a string frame in csv or textcell format into a frame block.
    `FrameBlock` | `convertToFrame(String input, int rows, int cols)` Converts an input string representation of a frame in textcell format into a frame block.
    `FrameBlock` | `convertToFrame(String input, String meta)` Converts an input string representation of a frame in csv or textcell format into a frame block.
    `MatrixBlock` | `convertToMatrix(InputStream input, int rows, int cols)` Converts an input stream of a string matrix in text format into a matrix block.
    `MatrixBlock` | `convertToMatrix(InputStream input, int rows, int cols, String format)` Converts an input stream of a string matrix in csv or text format into a matrix block.
    `MatrixBlock` | `convertToMatrix(InputStream input, String meta)` Converts an input stream of a string matrix in csv or textcell format into a matrix block.
    `MatrixBlock` | `convertToMatrix(String input, int rows, int cols)` Converts an input string representation of a matrix in textcell format into a matrix block.
    `MatrixBlock` | `convertToMatrix(String input, String meta)` Converts an input string representation of a matrix in csv or textcell format into a matrix block.
    `String[][]` | `convertToStringFrame(InputStream input, int rows, int cols)` Converts an input stream of a string frame in textcell format into a dense string array.
    `String[][]` | `convertToStringFrame(InputStream input, int rows, int cols, String format)` Converts an input stream of a string frame in csv or text format into a dense string array.
    `String[][]` | `convertToStringFrame(String input, int rows, int cols)` Converts an input stream of a string frame in textcell format into a dense string array.
    `String[][]` | `convertToStringFrame(String input, String meta)` Converts an input string representation of a frame in csv or textcell format into a dense string array.
    `void` | `gatherMemStats(boolean stats)` Sets a boolean flag indicating if memory profiling statistics should be gathered.
    `PreparedScript` | `prepareScript(String script, Map<String,String> nsscripts, Map<String,String> args, String[] inputs, String[] outputs)` Prepares (precompiles) a script, sets input parameter values, and registers input and output variables.
    `PreparedScript` | `prepareScript(String script, Map<String,String> args, String[] inputs, String[] outputs)` Prepares (precompiles) a script, sets input parameter values, and registers input and output variables.
    `PreparedScript` | `prepareScript(String script, String[] inputs, String[] outputs)` Prepares (precompiles) a script and registers input and output variables.
    `double[][]` | `readDoubleMatrix(String fname)` Reads an input matrix in arbitrary format from HDFS into a dense double array.
    `double[][]` | `readDoubleMatrix(String fname, Types.FileFormat fmt, long rows, long cols, int blen, long nnz)` Reads an input matrix in arbitrary format from HDFS into a dense double array.
    `String` | `readScript(String fname)` Read a DML or PyDML file as a string.
    `String[][]` | `readStringFrame(String fname)` Reads an input frame in arbitrary format from HDFS into a dense string array.
    `String[][]` | `readStringFrame(String fname, Types.FileFormat fmt, long rows, long cols)` Reads an input frame in arbitrary format from HDFS into a dense string array.
    `FrameBlock` | `readTransformMetaDataFromFile(String metapath)` Reads transform meta data from an HDFS file path and converts it into an in-memory FrameBlock object.
    `FrameBlock` | `readTransformMetaDataFromFile(String spec, String metapath)` Reads transform meta data from an HDFS file path and converts it into an in-memory FrameBlock object.
    `FrameBlock` | `readTransformMetaDataFromFile(String spec, String metapath, String colDelim)` Reads transform meta data from an HDFS file path and converts it into an in-memory FrameBlock object.
    `FrameBlock` | `readTransformMetaDataFromPath(String metapath)` Reads transform meta data from the class path and converts it into an in-memory FrameBlock object.
    `FrameBlock` | `readTransformMetaDataFromPath(String spec, String metapath)` Reads transform meta data from the class path and converts it into an in-memory FrameBlock object.
    `FrameBlock` | `readTransformMetaDataFromPath(String spec, String metapath, String colDelim)` Reads transform meta data from the class path and converts it into an in-memory FrameBlock object.
    `void` | `setLineage(boolean lt)` Sets a boolean flag indicating if lineage trace should be captured
    `void` | `setStatistics(boolean stats)` Sets a boolean flag indicating if runtime statistics should be gathered Same behavior as in "MLContext.setStatistics()"

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Constructor Detail

    - #### Connection

          public Connection()

      Connection constructor, the starting point for any other JMLC API calls.

    - #### Connection

          public Connection(CompilerConfig.ConfigType... cconfigs)

      Connection constructor, the starting point for any other JMLC API calls. This variant allows to enable a set of boolean compiler configurations.
      Parameters:
        `cconfigs` - one or many boolean compiler configurations to enable.

    - #### Connection

          public Connection(DMLConfig dmlconfig,
                            CompilerConfig.ConfigType... cconfigs)

      Connection constructor, the starting point for any other JMLC API calls. This variant allows to pass a global dml configuration and enable a set of boolean compiler configurations.
      Parameters:
        `dmlconfig` - a dml configuration.
        `cconfigs` - one or many boolean compiler configurations to enable.

    - #### Connection

          public Connection(DMLConfig dmlconfig)

      Connection constructor, the starting point for any other JMLC API calls. This variant allows to pass a global dml configuration.
      Parameters:
        `dmlconfig` - a dml configuration.

  - ### Method Detail

    - #### setStatistics

          public void setStatistics(boolean stats)

      Sets a boolean flag indicating if runtime statistics should be gathered Same behavior as in "MLContext.setStatistics()"
      Parameters:
        `stats` - boolean value with true indicating statistics should be gathered

    - #### setLineage

          public void setLineage(boolean lt)

      Sets a boolean flag indicating if lineage trace should be captured
      Parameters:
        `lt` - boolean value with true indicating lineage should be captured

    - #### gatherMemStats

          public void gatherMemStats(boolean stats)

      Sets a boolean flag indicating if memory profiling statistics should be gathered. The option is false by default.
      Parameters:
        `stats` - boolean value with true indicating memory statistics should be gathered

    - #### prepareScript

          public PreparedScript prepareScript(String script,
                                              String[] inputs,
                                              String[] outputs)

      Prepares (precompiles) a script and registers input and output variables.
      Parameters:
        `script` - string representing the DML or PyDML script
        `inputs` - string array of input variables to register
        `outputs` - string array of output variables to register
      Returns:
        PreparedScript object representing the precompiled script

    - #### prepareScript

          public PreparedScript prepareScript(String script,
                                              Map<String,String> args,
                                              String[] inputs,
                                              String[] outputs)

      Prepares (precompiles) a script, sets input parameter values, and registers input and output variables.
      Parameters:
        `script` - string representing the DML or PyDML script
        `args` - map of input parameters ($) and their values
        `inputs` - string array of input variables to register
        `outputs` - string array of output variables to register
      Returns:
        PreparedScript object representing the precompiled script

    - #### prepareScript

          public PreparedScript prepareScript(String script,
                                              Map<String,String> nsscripts,
                                              Map<String,String> args,
                                              String[] inputs,
                                              String[] outputs)

      Prepares (precompiles) a script, sets input parameter values, and registers input and output variables.
      Parameters:
        `script` - string representing of the DML or PyDML script
        `nsscripts` - map (name, script) of the DML or PyDML namespace scripts
        `args` - map of input parameters ($) and their values
        `inputs` - string array of input variables to register
        `outputs` - string array of output variables to register
      Returns:
        PreparedScript object representing the precompiled script

    - #### close

          public void close()

      Close connection to SystemDS, which clears the thread-local DML and compiler configurations.
      Specified by:
        `close` in interface `Closeable`
      Specified by:
        `close` in interface `AutoCloseable`

    - #### readScript

          public String readScript(String fname)
                            throws IOException

      Read a DML or PyDML file as a string.
      Parameters:
        `fname` - the filename of the script
      Returns:
        string content of the script file
      Throws:
        `IOException` - if IOException occurs

    - #### readDoubleMatrix

          public double[][] readDoubleMatrix(String fname)
                                      throws IOException

      Reads an input matrix in arbitrary format from HDFS into a dense double array. NOTE: this call currently only supports default configurations for CSV.
      Parameters:
        `fname` - the filename of the input matrix
      Returns:
        matrix as a two-dimensional double array
      Throws:
        `IOException` - if IOException occurs

    - #### readDoubleMatrix

          public double[][] readDoubleMatrix(String fname,
                                             Types.FileFormat fmt,
                                             long rows,
                                             long cols,
                                             int blen,
                                             long nnz)
                                      throws IOException

      Reads an input matrix in arbitrary format from HDFS into a dense double array. NOTE: this call currently only supports default configurations for CSV.
      Parameters:
        `fname` - the filename of the input matrix
        `fmt` - file format
        `rows` - number of rows in the matrix
        `cols` - number of columns in the matrix
        `blen` - block length
        `nnz` - number of non-zero values, -1 indicates unknown
      Returns:
        matrix as a two-dimensional double array
      Throws:
        `IOException` - if IOException occurs

    - #### convertToDoubleMatrix

          public double[][] convertToDoubleMatrix(String input,
                                                  String meta)
                                           throws IOException

      Converts an input string representation of a matrix in csv or textcell format into a dense double array. The meta data string is the SystemDS generated .mtd file including the number of rows and columns.
      Parameters:
        `input` - string matrix in csv or textcell format
        `meta` - string representing SystemDS matrix metadata in JSON format
      Returns:
        matrix as a two-dimensional double array
      Throws:
        `IOException` - if IOException occurs

    - #### convertToDoubleMatrix

          public double[][] convertToDoubleMatrix(String input,
                                                  int rows,
                                                  int cols)
                                           throws IOException

      Converts an input string representation of a matrix in textcell format into a dense double array.
      Parameters:
        `input` - string matrix in textcell format
        `rows` - number of rows in the matrix
        `cols` - number of columns in the matrix
      Returns:
        matrix as a two-dimensional double array
      Throws:
        `IOException` - if IOException occurs

    - #### convertToDoubleMatrix

          public double[][] convertToDoubleMatrix(InputStream input,
                                                  int rows,
                                                  int cols)
                                           throws IOException

      Converts an input stream of a string matrix in textcell format into a dense double array.
      Parameters:
        `input` - InputStream to a string matrix in textcell format
        `rows` - number of rows in the matrix
        `cols` - number of columns in the matrix
      Returns:
        matrix as a two-dimensional double array
      Throws:
        `IOException` - if IOException occurs

    - #### convertToDoubleMatrix

          public double[][] convertToDoubleMatrix(InputStream input,
                                                  int rows,
                                                  int cols,
                                                  String format)
                                           throws IOException

      Converts an input stream of a string matrix in csv or textcell format into a dense double array.
      Parameters:
        `input` - InputStream to a string matrix in csv or textcell format
        `rows` - number of rows in the matrix
        `cols` - number of columns in the matrix
        `format` - input format of the given stream
      Returns:
        matrix as a two-dimensional double array
      Throws:
        `IOException` - if IOException occurs

    - #### convertToMatrix

          public MatrixBlock convertToMatrix(String input,
                                             String meta)
                                      throws IOException

      Converts an input string representation of a matrix in csv or textcell format into a matrix block. The meta data string is the SystemDS generated .mtd file including the number of rows and columns.
      Parameters:
        `input` - string matrix in csv or textcell format
        `meta` - string representing SystemDS matrix metadata in JSON format
      Returns:
        matrix as a matrix block
      Throws:
        `IOException` - if IOException occurs

    - #### convertToMatrix

          public MatrixBlock convertToMatrix(InputStream input,
                                             String meta)
                                      throws IOException

      Converts an input stream of a string matrix in csv or textcell format into a matrix block. The meta data string is the SystemDS generated .mtd file including the number of rows and columns.
      Parameters:
        `input` - InputStream to a string matrix in csv or textcell format
        `meta` - string representing SystemDS matrix metadata in JSON format
      Returns:
        matrix as a matrix block
      Throws:
        `IOException` - if IOException occurs

    - #### convertToMatrix

          public MatrixBlock convertToMatrix(String input,
                                             int rows,
                                             int cols)
                                      throws IOException

      Converts an input string representation of a matrix in textcell format into a matrix block.
      Parameters:
        `input` - string matrix in textcell format
        `rows` - number of rows in the matrix
        `cols` - number of columns in the matrix
      Returns:
        matrix as a matrix block
      Throws:
        `IOException` - if IOException occurs

    - #### convertToMatrix

          public MatrixBlock convertToMatrix(InputStream input,
                                             int rows,
                                             int cols)
                                      throws IOException

      Converts an input stream of a string matrix in text format into a matrix block.
      Parameters:
        `input` - InputStream to a string matrix in text format
        `rows` - number of rows in the matrix
        `cols` - number of columns in the matrix
      Returns:
        matrix as a matrix block
      Throws:
        `IOException` - if IOException occurs

    - #### convertToMatrix

          public MatrixBlock convertToMatrix(InputStream input,
                                             int rows,
                                             int cols,
                                             String format)
                                      throws IOException

      Converts an input stream of a string matrix in csv or text format into a matrix block.
      Parameters:
        `input` - InputStream to a string matrix in csv or text format
        `rows` - number of rows in the matrix
        `cols` - number of columns in the matrix
        `format` - input format of the given stream
      Returns:
        matrix as a matrix block
      Throws:
        `IOException` - if IOException occurs

    - #### readStringFrame

          public String[][] readStringFrame(String fname)
                                     throws IOException

      Reads an input frame in arbitrary format from HDFS into a dense string array. NOTE: this call currently only supports default configurations for CSV.
      Parameters:
        `fname` - the filename of the input frame
      Returns:
        frame as a two-dimensional string array
      Throws:
        `IOException` - if IOException occurs

    - #### readStringFrame

          public String[][] readStringFrame(String fname,
                                            Types.FileFormat fmt,
                                            long rows,
                                            long cols)
                                     throws IOException

      Reads an input frame in arbitrary format from HDFS into a dense string array. NOTE: this call currently only supports default configurations for CSV.
      Parameters:
        `fname` - the filename of the input frame
        `fmt` - file format type
        `rows` - number of rows in the frame
        `cols` - number of columns in the frame
      Returns:
        frame as a two-dimensional string array
      Throws:
        `IOException` - if IOException occurs

    - #### convertToStringFrame

          public String[][] convertToStringFrame(String input,
                                                 String meta)
                                          throws IOException

      Converts an input string representation of a frame in csv or textcell format into a dense string array. The meta data string is the SystemDS generated .mtd file including the number of rows and columns.
      Parameters:
        `input` - string frame in csv or textcell format
        `meta` - string representing SystemDS frame metadata in JSON format
      Returns:
        frame as a two-dimensional string array
      Throws:
        `IOException` - if IOException occurs

    - #### convertToStringFrame

          public String[][] convertToStringFrame(String input,
                                                 int rows,
                                                 int cols)
                                          throws IOException

      Converts an input stream of a string frame in textcell format into a dense string array.
      Parameters:
        `input` - string frame in textcell format
        `rows` - number of rows in the frame
        `cols` - number of columns in the frame
      Returns:
        frame as a two-dimensional string array
      Throws:
        `IOException` - if IOException occurs

    - #### convertToStringFrame

          public String[][] convertToStringFrame(InputStream input,
                                                 int rows,
                                                 int cols)
                                          throws IOException

      Converts an input stream of a string frame in textcell format into a dense string array.
      Parameters:
        `input` - InputStream to a string frame in text format
        `rows` - number of rows in the frame
        `cols` - number of columns in the frame
      Returns:
        frame as a two-dimensional string array
      Throws:
        `IOException` - if IOException occurs

    - #### convertToStringFrame

          public String[][] convertToStringFrame(InputStream input,
                                                 int rows,
                                                 int cols,
                                                 String format)
                                          throws IOException

      Converts an input stream of a string frame in csv or text format into a dense string array.
      Parameters:
        `input` - InputStream to a string frame in csv or text format
        `rows` - number of rows in the frame
        `cols` - number of columns in the frame
        `format` - input format of the given stream
      Returns:
        frame as a two-dimensional string array
      Throws:
        `IOException` - if IOException occurs

    - #### convertToFrame

          public FrameBlock convertToFrame(String input,
                                           String meta)
                                    throws IOException

      Converts an input string representation of a frame in csv or textcell format into a frame block. The meta data string is the SystemDS generated .mtd file including the number of rows and columns.
      Parameters:
        `input` - string frame in csv or textcell format
        `meta` - string representing SystemDS frame metadata in JSON format
      Returns:
        frame as a frame block
      Throws:
        `IOException` - if IOException occurs

    - #### convertToFrame

          public FrameBlock convertToFrame(InputStream input,
                                           String meta)
                                    throws IOException

      Converts an input stream of a string frame in csv or textcell format into a frame block. The meta data string is the SystemDS generated .mtd file including the number of rows and columns.
      Parameters:
        `input` - InputStream to a string frame in csv or textcell format
        `meta` - string representing SystemDS frame metadata in JSON format
      Returns:
        frame as a frame block
      Throws:
        `IOException` - if IOException occurs

    - #### convertToFrame

          public FrameBlock convertToFrame(String input,
                                           int rows,
                                           int cols)
                                    throws IOException

      Converts an input string representation of a frame in textcell format into a frame block.
      Parameters:
        `input` - string frame in textcell format
        `rows` - number of rows in the frame
        `cols` - number of columns in the frame
      Returns:
        frame as a frame block
      Throws:
        `IOException` - if IOException occurs

    - #### convertToFrame

          public FrameBlock convertToFrame(InputStream input,
                                           int rows,
                                           int cols)
                                    throws IOException

      Converts an input stream of a string frame in textcell format into a frame block.
      Parameters:
        `input` - InputStream to a string frame in textcell format
        `rows` - number of rows in the frame
        `cols` - number of columns in the frame
      Returns:
        frame as a frame block
      Throws:
        `IOException` - if IOException occurs

    - #### convertToFrame

          public FrameBlock convertToFrame(InputStream input,
                                           int rows,
                                           int cols,
                                           String format)
                                    throws IOException

      Converts an input stream of a frame in csv or textcell format into a frame block.
      Parameters:
        `input` - InputStream to a string frame in csv or textcell format
        `rows` - number of rows in the frame
        `cols` - number of columns in the frame
        `format` - input format of the given stream
      Returns:
        frame as a frame block
      Throws:
        `IOException` - if IOException occurs

    - #### readTransformMetaDataFromFile

          public FrameBlock readTransformMetaDataFromFile(String metapath)
                                                   throws IOException

      Reads transform meta data from an HDFS file path and converts it into an in-memory FrameBlock object. The column names in the meta data file 'column.names' is processed with default separator ','.
      Parameters:
        `metapath` - hdfs file path to meta data directory
      Returns:
        FrameBlock object representing transform metadata
      Throws:
        `IOException` - if IOException occurs

    - #### readTransformMetaDataFromFile

          public FrameBlock readTransformMetaDataFromFile(String spec,
                                                          String metapath)
                                                   throws IOException

      Reads transform meta data from an HDFS file path and converts it into an in-memory FrameBlock object. The column names in the meta data file 'column.names' is processed with default separator ','.
      Parameters:
        `spec` - transform specification as json string
        `metapath` - hdfs file path to meta data directory
      Returns:
        FrameBlock object representing transform metadata
      Throws:
        `IOException` - if IOException occurs

    - #### readTransformMetaDataFromFile

          public FrameBlock readTransformMetaDataFromFile(String spec,
                                                          String metapath,
                                                          String colDelim)
                                                   throws IOException

      Reads transform meta data from an HDFS file path and converts it into an in-memory FrameBlock object.
      Parameters:
        `spec` - transform specification as json string
        `metapath` - hdfs file path to meta data directory
        `colDelim` - separator for processing column names in the meta data file 'column.names'
      Returns:
        FrameBlock object representing transform metadata
      Throws:
        `IOException` - if IOException occurs

    - #### readTransformMetaDataFromPath

          public FrameBlock readTransformMetaDataFromPath(String metapath)
                                                   throws IOException

      Reads transform meta data from the class path and converts it into an in-memory FrameBlock object. The column names in the meta data file 'column.names' is processed with default separator ','.
      Parameters:
        `metapath` - resource path to meta data directory
      Returns:
        FrameBlock object representing transform metadata
      Throws:
        `IOException` - if IOException occurs

    - #### readTransformMetaDataFromPath

          public FrameBlock readTransformMetaDataFromPath(String spec,
                                                          String metapath)
                                                   throws IOException

      Reads transform meta data from the class path and converts it into an in-memory FrameBlock object. The column names in the meta data file 'column.names' is processed with default separator ','.
      Parameters:
        `spec` - transform specification as json string
        `metapath` - resource path to meta data directory
      Returns:
        FrameBlock object representing transform metadata
      Throws:
        `IOException` - if IOException occurs

    - #### readTransformMetaDataFromPath

          public FrameBlock readTransformMetaDataFromPath(String spec,
                                                          String metapath,
                                                          String colDelim)
                                                   throws IOException

      Reads transform meta data from the class path and converts it into an in-memory FrameBlock object.
      Parameters:
        `spec` - transform specification as json string
        `metapath` - resource path to meta data directory
        `colDelim` - separator for processing column names in the meta data file 'column.names'
      Returns:
        FrameBlock object representing transform metadata
      Throws:
        `IOException` - if IOException occurs

Copyright © 2021 The Apache Software Foundation. All rights reserved.