Package org.apache.sysds.runtime.instructions.gpu.context

Class GPUContext

java.lang.Object
org.apache.sysds.runtime.instructions.gpu.context.GPUContext
public class GPUContext extends Object
Represents a context per GPU accessible through the same JVM. Each context holds cublas, cusparse, cudnn... handles which are separate for each GPU.

  • Method Details

    • getMemoryManager

      public GPUMemoryManager getMemoryManager()

    • cudaGetDevice

      public static int cudaGetDevice()
      Returns which device is currently being used.
      Returns:
      the current device for the calling host thread

    • printMemoryInfo

      public void printMemoryInfo(String opcode)
      Print information of memory usage.
      Parameters:
      opcode - opcode of caller

    • getDeviceNum

      public int getDeviceNum()
      Returns which device is assigned to this GPUContext instance.
      Returns:
      active device assigned to this GPUContext instance

    • initializeThread

      public void initializeThread()
      Sets the device for the calling thread. This method must be called after ExecutionContext.getGPUContext(int) If in a multithreaded environment like parfor, this method must be called when in the appropriate thread.

    • allocate

      public jcuda.Pointer allocate(String instructionName, long size, boolean initialize)
      Invokes memory manager's malloc method
      Parameters:
      instructionName - name of instruction for which to record per instruction performance statistics, null if you don't want to record
      size - size of data (in bytes) to allocate
      initialize - if cudaMemset() should be called
      Returns:
      jcuda pointer

    • allocate

      public jcuda.Pointer allocate(String instructionName, long size)
      Default behavior for gpu memory allocation (init to zero)
      Parameters:
      instructionName - Name of the instruction calling allocate
      size - size in bytes
      Returns:
      jcuda pointer

    • cudaFreeHelper

      public void cudaFreeHelper(String instructionName, jcuda.Pointer toFree, boolean eager)
      Does cudaFree calls, lazily.
      Parameters:
      instructionName - name of the instruction for which to record per instruction free time, null if you do not want to record
      toFree - Pointer instance to be freed
      eager - true if to be done eagerly

    • getAvailableMemory

      public long getAvailableMemory()
      Gets the available memory on GPU that SystemDS can use.
      Returns:
      the available memory in bytes

    • ensureComputeCapability

      public void ensureComputeCapability()
      Makes sure that GPU that SystemDS is trying to use has the minimum compute capability needed.

    • createGPUObject

      public GPUObject createGPUObject(MatrixObject mo)
      Instantiates a new GPUObject initialized with the given MatrixObject.
      Parameters:
      mo - a MatrixObject that represents a matrix
      Returns:
      a new GPUObject instance

    • shallowCopyGPUObject

      public GPUObject shallowCopyGPUObject(GPUObject source, MatrixObject mo)
      Shallow copy the given source GPUObject to a new GPUObject and assign that to the given MatrixObject. This copy doesn't memcopy the device memory.
      Parameters:
      source - a GPUObject which is the source of the copy
      mo - a MatrixObject to associate with the new GPUObject
      Returns:
      a new GPUObject instance

    • getGPUProperties

      public jcuda.runtime.cudaDeviceProp getGPUProperties()
      Gets the device properties for the active GPU (set with cudaSetDevice()).
      Returns:
      the device properties

    • getMaxThreadsPerBlock

      public int getMaxThreadsPerBlock()
      Gets the maximum number of threads per block for "active" GPU.
      Returns:
      the maximum number of threads per block

    • getMaxBlocks

      public int getMaxBlocks()
      Gets the maximum number of blocks supported by the active cuda device.
      Returns:
      the maximum number of blocks supported

    • getMaxSharedMemory

      public long getMaxSharedMemory()
      Gets the shared memory per block supported by the active cuda device.
      Returns:
      the shared memory per block

    • getWarpSize

      public int getWarpSize()
      Gets the warp size supported by the active cuda device.
      Returns:
      the warp size

    • getCudnnHandle

      public jcuda.jcudnn.cudnnHandle getCudnnHandle()
      Returns the cudnnHandle for Deep Neural Network operations on the GPU.
      Returns:
      cudnnHandle for current thread

    • getCublasHandle

      public jcuda.jcublas.cublasHandle getCublasHandle()
      Returns cublasHandle for BLAS operations on the GPU.
      Returns:
      cublasHandle for current thread

    • getCusparseHandle

      public jcuda.jcusparse.cusparseHandle getCusparseHandle()
      Returns cusparseHandle for certain sparse BLAS operations on the GPU.
      Returns:
      cusparseHandle for current thread

    • getCusolverDnHandle

      public jcuda.jcusolver.cusolverDnHandle getCusolverDnHandle()
      Returns cusolverDnHandle for invoking solve() function on dense matrices on the GPU.
      Returns:
      cusolverDnHandle for current thread

    • getKernels

      public JCudaKernels getKernels()
      Returns utility class used to launch custom CUDA kernel, specific to the active GPU for this GPUContext.
      Returns:
      JCudaKernels for current thread

    • destroy

      public void destroy()
      Destroys this GPUContext object.

    • clearMemory

      public void clearMemory()
      Clears all memory used by this GPUContext. Be careful to ensure that no memory is currently being used in the temporary memory before invoking this. If memory is being used between MLContext invocations, they are pointed to by a GPUObject instance which would be part of the MatrixObject. The cleanup of that MatrixObject instance will cause the memory associated with that block on the GPU to be freed up.

    • clearTemporaryMemory

      public void clearTemporaryMemory()

    • toString

      public String toString()
      Overrides:
      toString in class Object