Package org.apache.sysds.runtime.instructions.gpu.context

## Class GPUContext

- java.lang.Object
- - org.apache.sysds.runtime.instructions.gpu.context.GPUContext

-     public class GPUContext
      extends Object

  Represents a context per GPU accessible through the same JVM. Each context holds cublas, cusparse, cudnn... handles which are separate for each GPU.

- - ### Method Summary

    All Methods Static Methods Instance Methods Concrete Methods
    Modifier and Type | Method | Description
    --- | --- | ---
    `jcuda.Pointer` | `allocate​(String instructionName, long size)` | Default behavior for gpu memory allocation (init to zero)
    `jcuda.Pointer` | `allocate​(String instructionName, long size, boolean initialize)` | Invokes memory manager's malloc method
    `void` | `clearMemory()` | Clears all memory used by this `GPUContext`.
    `void` | `clearTemporaryMemory()` |
    `GPUObject` | `createGPUObject​(MatrixObject mo)` | Instantiates a new `GPUObject` initialized with the given `MatrixObject`.
    `void` | `cudaFreeHelper​(String instructionName, jcuda.Pointer toFree, boolean eager)` | Does cudaFree calls, lazily.
    `static int` | `cudaGetDevice()` | Returns which device is currently being used.
    `void` | `destroy()` | Destroys this GPUContext object.
    `void` | `ensureComputeCapability()` | Makes sure that GPU that SystemDS is trying to use has the minimum compute capability needed.
    `long` | `getAvailableMemory()` | Gets the available memory on GPU that SystemDS can use.
    `jcuda.jcublas.cublasHandle` | `getCublasHandle()` | Returns cublasHandle for BLAS operations on the GPU.
    `jcuda.jcudnn.cudnnHandle` | `getCudnnHandle()` | Returns the cudnnHandle for Deep Neural Network operations on the GPU.
    `jcuda.jcusolver.cusolverDnHandle` | `getCusolverDnHandle()` | Returns cusolverDnHandle for invoking solve() function on dense matrices on the GPU.
    `jcuda.jcusparse.cusparseHandle` | `getCusparseHandle()` | Returns cusparseHandle for certain sparse BLAS operations on the GPU.
    `int` | `getDeviceNum()` | Returns which device is assigned to this GPUContext instance.
    `jcuda.runtime.cudaDeviceProp` | `getGPUProperties()` | Gets the device properties for the active GPU (set with cudaSetDevice()).
    `JCudaKernels` | `getKernels()` | Returns utility class used to launch custom CUDA kernel, specific to the active GPU for this GPUContext.
    `int` | `getMaxBlocks()` | Gets the maximum number of blocks supported by the active cuda device.
    `long` | `getMaxSharedMemory()` | Gets the shared memory per block supported by the active cuda device.
    `int` | `getMaxThreadsPerBlock()` | Gets the maximum number of threads per block for "active" GPU.
    `GPUMemoryManager` | `getMemoryManager()` |
    `int` | `getWarpSize()` | Gets the warp size supported by the active cuda device.
    `void` | `initializeThread()` | Sets the device for the calling thread.
    `void` | `printMemoryInfo​(String opcode)` | Print information of memory usage.
    `GPUObject` | `shallowCopyGPUObject​(GPUObject source, MatrixObject mo)` | Shallow copy the given source `GPUObject` to a new `GPUObject` and assign that to the given `MatrixObject`.
    `String` | `toString()` |

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, wait, wait, wait`

- - ### Method Detail

    - #### getMemoryManager

          public GPUMemoryManager getMemoryManager()

    - #### cudaGetDevice

          public static int cudaGetDevice()

      Returns which device is currently being used.
      Returns:
        the current device for the calling host thread

    - #### printMemoryInfo

          public void printMemoryInfo​(String opcode)

      Print information of memory usage.
      Parameters:
        `opcode` - opcode of caller

    - #### getDeviceNum

          public int getDeviceNum()

      Returns which device is assigned to this GPUContext instance.
      Returns:
        active device assigned to this GPUContext instance

    - #### initializeThread

          public void initializeThread()

      Sets the device for the calling thread. This method must be called after `ExecutionContext.getGPUContext(int)` If in a multithreaded environment like parfor, this method must be called when in the appropriate thread.

    - #### allocate

          public jcuda.Pointer allocate​(String instructionName,
                                        long size,
                                        boolean initialize)

      Invokes memory manager's malloc method
      Parameters:
        `instructionName` - name of instruction for which to record per instruction performance statistics, null if you don't want to record
        `size` - size of data (in bytes) to allocate
        `initialize` - if cudaMemset() should be called
      Returns:
        jcuda pointer

    - #### allocate

          public jcuda.Pointer allocate​(String instructionName,
                                        long size)

      Default behavior for gpu memory allocation (init to zero)
      Parameters:
        `instructionName` - Name of the instruction calling allocate
        `size` - size in bytes
      Returns:
        jcuda pointer

    - #### cudaFreeHelper

          public void cudaFreeHelper​(String instructionName,
                                     jcuda.Pointer toFree,
                                     boolean eager)

      Does cudaFree calls, lazily.
      Parameters:
        `instructionName` - name of the instruction for which to record per instruction free time, null if you do not want to record
        `toFree` - `Pointer` instance to be freed
        `eager` - true if to be done eagerly

    - #### getAvailableMemory

          public long getAvailableMemory()

      Gets the available memory on GPU that SystemDS can use.
      Returns:
        the available memory in bytes

    - #### ensureComputeCapability

          public void ensureComputeCapability()

      Makes sure that GPU that SystemDS is trying to use has the minimum compute capability needed.

    - #### createGPUObject

          public GPUObject createGPUObject​(MatrixObject mo)

      Instantiates a new `GPUObject` initialized with the given `MatrixObject`.
      Parameters:
        `mo` - a `MatrixObject` that represents a matrix
      Returns:
        a new `GPUObject` instance

    - #### shallowCopyGPUObject

          public GPUObject shallowCopyGPUObject​(GPUObject source,
                                                MatrixObject mo)

      Shallow copy the given source `GPUObject` to a new `GPUObject` and assign that to the given `MatrixObject`. This copy doesn't memcopy the device memory.
      Parameters:
        `source` - a `GPUObject` which is the source of the copy
        `mo` - a `MatrixObject` to associate with the new `GPUObject`
      Returns:
        a new `GPUObject` instance

    - #### getGPUProperties

          public jcuda.runtime.cudaDeviceProp getGPUProperties()

      Gets the device properties for the active GPU (set with cudaSetDevice()).
      Returns:
        the device properties

    - #### getMaxThreadsPerBlock

          public int getMaxThreadsPerBlock()

      Gets the maximum number of threads per block for "active" GPU.
      Returns:
        the maximum number of threads per block

    - #### getMaxBlocks

          public int getMaxBlocks()

      Gets the maximum number of blocks supported by the active cuda device.
      Returns:
        the maximum number of blocks supported

    - #### getMaxSharedMemory

          public long getMaxSharedMemory()

      Gets the shared memory per block supported by the active cuda device.
      Returns:
        the shared memory per block

    - #### getWarpSize

          public int getWarpSize()

      Gets the warp size supported by the active cuda device.
      Returns:
        the warp size

    - #### getCudnnHandle

          public jcuda.jcudnn.cudnnHandle getCudnnHandle()

      Returns the cudnnHandle for Deep Neural Network operations on the GPU.
      Returns:
        cudnnHandle for current thread

    - #### getCublasHandle

          public jcuda.jcublas.cublasHandle getCublasHandle()

      Returns cublasHandle for BLAS operations on the GPU.
      Returns:
        cublasHandle for current thread

    - #### getCusparseHandle

          public jcuda.jcusparse.cusparseHandle getCusparseHandle()

      Returns cusparseHandle for certain sparse BLAS operations on the GPU.
      Returns:
        cusparseHandle for current thread

    - #### getCusolverDnHandle

          public jcuda.jcusolver.cusolverDnHandle getCusolverDnHandle()

      Returns cusolverDnHandle for invoking solve() function on dense matrices on the GPU.
      Returns:
        cusolverDnHandle for current thread

    - #### getKernels

          public JCudaKernels getKernels()

      Returns utility class used to launch custom CUDA kernel, specific to the active GPU for this GPUContext.
      Returns:
        `JCudaKernels` for current thread

    - #### destroy

          public void destroy()

      Destroys this GPUContext object.

    - #### clearMemory

          public void clearMemory()

      Clears all memory used by this `GPUContext`. Be careful to ensure that no memory is currently being used in the temporary memory before invoking this. If memory is being used between MLContext invocations, they are pointed to by a `GPUObject` instance which would be part of the `MatrixObject`. The cleanup of that `MatrixObject` instance will cause the memory associated with that block on the GPU to be freed up.

    - #### clearTemporaryMemory

          public void clearTemporaryMemory()

    - #### toString

          public String toString()

      Overrides:
        `toString` in class `Object`