Package org.pentaho.di.core.compress.hadoopsnappy

Class HadoopSnappyCompressionProvider

java.lang.Object
org.pentaho.di.core.compress.hadoopsnappy.HadoopSnappyCompressionProvider
All Implemented Interfaces:
CompressionProvider
public class HadoopSnappyCompressionProvider extends Object implements CompressionProvider

  • Field Details

    • IO_COMPRESSION_CODEC_SNAPPY_DEFAULT_BUFFERSIZE

      public static final int IO_COMPRESSION_CODEC_SNAPPY_DEFAULT_BUFFERSIZE
      See Also:

  • Constructor Details

    • HadoopSnappyCompressionProvider

      public HadoopSnappyCompressionProvider()

  • Method Details

    • getActiveSnappyShim

      public static Object getActiveSnappyShim() throws Exception
      Locate the Snappy Shim for the active Hadoop Configuration via the Hadoop Configuration Util
      Returns:
      A org.pentaho.hadoop.shim.spi.SnappyShim to interact with Snappy
      Throws:
      Exception - Error locating a valid Snappy shim:

      • org.pentaho.hadoop.hive.jdbc.HadoopConfigurationUtil could not be located
      • No active Hadoop configuration
      • Active Hadoop configuration doesn't support Snappy

    • isHadoopSnappyAvailable

      public static boolean isHadoopSnappyAvailable()
      Tests whether hadoop-snappy (not to be confused with other java-based snappy implementations such as jsnappy or snappy-java) plus the native snappy libraries are available.
      Returns:
      true if hadoop-snappy is available on the classpath

    • createInputStream

      public HadoopSnappyCompressionInputStream createInputStream(InputStream in) throws IOException
      Description copied from interface: CompressionProvider
      Creates an input stream that decompresses data (according to the provider's implementation)
      Specified by:
      createInputStream in interface CompressionProvider
      Parameters:
      in - An existing input stream to wrap
      Returns:
      a CompressionInputStream object that decompresses incoming data
      Throws:
      IOException

    • supportsInput

      public boolean supportsInput()
      Description copied from interface: CompressionProvider
      Whether this compression provider supports input streams
      Specified by:
      supportsInput in interface CompressionProvider
      Returns:
      true if the provider supports input streams, false otherwise

    • createOutputStream

      public HadoopSnappyCompressionOutputStream createOutputStream(OutputStream out) throws IOException
      Description copied from interface: CompressionProvider
      Creates an output stream that compresses data (according to the provider's implementation)
      Specified by:
      createOutputStream in interface CompressionProvider
      Parameters:
      out - An existing output stream to wrap
      Returns:
      a CompressionOutputStream object that compresses outgoing data
      Throws:
      IOException

    • supportsOutput

      public boolean supportsOutput()
      Description copied from interface: CompressionProvider
      Whether this compression provider supports output streams
      Specified by:
      supportsOutput in interface CompressionProvider
      Returns:
      true if the provider supports output streams, false otherwise

    • getDescription

      public String getDescription()
      Description copied from interface: CompressionProvider
      Gets the name of this provider. Used for display e.g.
      Specified by:
      getDescription in interface CompressionProvider
      Returns:
      A String containing a description of this provider

    • getName

      public String getName()
      Description copied from interface: CompressionProvider
      Gets the name of this provider. Used for display and as a reference in saved artifacts (transformations, e.g.)
      Specified by:
      getName in interface CompressionProvider
      Returns:
      A String containing the name of this provider

    • getDefaultExtension

      public String getDefaultExtension()
      Description copied from interface: CompressionProvider
      Gets the default file extension for this provider. If the streams are wrapped in File streams, this method can be used to determine an appropriate extension to append to the filename so the file will be recognized as an artifact of the compression mechanism (.zip, .bz2, e.g.)
      Specified by:
      getDefaultExtension in interface CompressionProvider
      Returns:
      A String containing the default file extension for this provider