Package org.apache.sysds.runtime.util

Class LocalFileUtils

java.lang.Object
org.apache.sysds.runtime.util.LocalFileUtils
public class LocalFileUtils extends Object

  • Field Details

  • Constructor Details

    • LocalFileUtils

      public LocalFileUtils()

  • Method Details

    • readMatrixBlockFromLocal

      public static MatrixBlock readMatrixBlockFromLocal(String fname) throws IOException
      Reads a matrix block from local file system.
      Parameters:
      fname - file name to read
      Returns:
      matrix block
      Throws:
      IOException - if IOException occurs

    • readMatrixBlockFromLocal

      public static MatrixBlock readMatrixBlockFromLocal(String fname, MatrixBlock reuse) throws IOException
      Reads a matrix block from local file system.
      Parameters:
      fname - file name to read
      reuse - matrix block to reuse
      Returns:
      matrix block
      Throws:
      IOException - if IOException occurs

    • readFrameBlockFromLocal

      public static FrameBlock readFrameBlockFromLocal(String fname) throws IOException
      Reads a frame block from local file system.
      Parameters:
      fname - file name to read
      Returns:
      frame block
      Throws:
      IOException - if IOException occurs

    • readCacheBlockFromLocal

      public static CacheBlock<?> readCacheBlockFromLocal(String fname, boolean matrix) throws IOException
      Reads a matrix/frame block from local file system.
      Parameters:
      fname - file name to read
      matrix - if true, read matrix. if false, read frame.
      Returns:
      cache block (common interface to MatrixBlock and FrameBlock)
      Throws:
      IOException - if IOException occurs

    • readWritableFromLocal

      public static org.apache.hadoop.io.Writable readWritableFromLocal(String fname, org.apache.hadoop.io.Writable ret) throws IOException
      Reads an arbitrary writable from local file system, using a fused buffered reader with special support for matrix blocks.
      Parameters:
      fname - file name to read
      ret - hadoop writable
      Returns:
      hadoop writable
      Throws:
      IOException - if IOException occurs

    • readMatrixBlockFromStream

      public static MatrixBlock readMatrixBlockFromStream(InputStream is) throws IOException
      Reads a matrix block from an input stream, using a fused buffered reader with special support for matrix blocks.
      Parameters:
      is - input stream to read
      Returns:
      matrix block
      Throws:
      IOException - if IOException occurs

    • readFrameBlockFromStream

      public static FrameBlock readFrameBlockFromStream(InputStream is) throws IOException
      Reads a frame block from an input stream, using a fused buffered reader with special support for matrix blocks.
      Parameters:
      is - input stream to read
      Returns:
      frame block
      Throws:
      IOException - if IOException occurs

    • readWritableFromStream

      public static org.apache.hadoop.io.Writable readWritableFromStream(InputStream is, org.apache.hadoop.io.Writable ret) throws IOException
      Reads an arbitrary writable from an input stream, using a fused buffered reader with special support for matrix blocks.
      Parameters:
      is - input stream to read
      ret - hadoop writable
      Returns:
      hadoop writable
      Throws:
      IOException - if IOException occurs

    • writeMatrixBlockToLocal

      public static void writeMatrixBlockToLocal(String fname, MatrixBlock mb) throws IOException
      Writes a matrix block to local file system.
      Parameters:
      fname - file name to write
      mb - matrix block
      Throws:
      IOException - if IOException occurs

    • writeFrameBlockToLocal

      public static void writeFrameBlockToLocal(String fname, FrameBlock fb) throws IOException
      Writes a frame block to local file system.
      Parameters:
      fname - file name to write
      fb - fame block
      Throws:
      IOException - if IOException occurs

    • writeCacheBlockToLocal

      public static void writeCacheBlockToLocal(String fname, CacheBlock<?> cb) throws IOException
      Writes a matrix/frame block to local file system.
      Parameters:
      fname - file name to write
      cb - cache block (common interface to matrix block and frame block)
      Throws:
      IOException - if IOException occurs

    • writeWritableToLocal

      public static void writeWritableToLocal(String fname, org.apache.hadoop.io.Writable mb, boolean doubleBuffering) throws IOException
      Writes an arbitrary writable to local file system, using a fused buffered writer with special support for matrix blocks.
      Parameters:
      fname - file name to write
      mb - Hadoop writable
      doubleBuffering - overlay serialization and I/O
      Throws:
      IOException - if IOException occurs

    • writeByteArrayToLocal

      public static void writeByteArrayToLocal(String fname, byte[] data) throws IOException
      Throws:
      IOException

    • readBlockSequenceFromLocal

      public static int readBlockSequenceFromLocal(String fname, Pair<MatrixIndexes,MatrixValue>[] outValues, HashMap<MatrixIndexes,Integer> outMap) throws IOException
      Throws:
      IOException

    • writeBlockSequenceToLocal

      public static void writeBlockSequenceToLocal(String fname, Pair<MatrixIndexes,MatrixValue>[] inValues, int len) throws IOException
      Throws:
      IOException

    • createLocalFileIfNotExist

      public static boolean createLocalFileIfNotExist(String dir)

    • deleteFileIfExists

      public static void deleteFileIfExists(String dir)

    • deleteFileIfExists

      public static void deleteFileIfExists(String dir, boolean fileOnly)

    • isExisting

      public static boolean isExisting(String dir)

    • createLocalFileIfNotExist

      public static boolean createLocalFileIfNotExist(String dir, String permission)

    • setLocalFilePermissions

      public static void setLocalFilePermissions(File file, String permissions)

    • checkAndCreateStagingDir

      public static String checkAndCreateStagingDir(String dir)

    • createWorkingDirectory

      public static String createWorkingDirectory()

    • createWorkingDirectoryWithUUID

      public static String createWorkingDirectoryWithUUID(String uuid)

    • cleanupWorkingDirectory

      public static void cleanupWorkingDirectory()

    • cleanupWorkingDirectory

      public static void cleanupWorkingDirectory(String dir)

    • cleanupRcWorkingDirectory

      public static int cleanupRcWorkingDirectory(String dir)

    • rDelete

      public static void rDelete(File dir)
      Recursively deletes an entire local file system directory.
      Parameters:
      dir - directory to delete recursively

    • rcDelete

      public static int rcDelete(File dir)
      Recursively deletes an entire local file system directory and returns the number of files deleted.
      Parameters:
      dir - directory to delete recursively
      Returns:
      number of files deleted

    • getWorkingDir

      public static String getWorkingDir(String category)

    • getUniqueWorkingDir

      public static String getUniqueWorkingDir(String category)

    • writeTextFile

      public static void writeTextFile(File file, String text) throws IOException
      Writes a simple text file to local file system.
      Parameters:
      file - output file
      text - content of text file
      Throws:
      IOException - errors in file open/write/close