org.pentaho.di.core.compress.hadoopsnappy

Class HadoopSnappyCompressionProvider

java.lang.Object

org.pentaho.di.core.compress.hadoopsnappy.HadoopSnappyCompressionProvider

All Implemented Interfaces:

CompressionProvider

public class HadoopSnappyCompressionProvider
extends Object
implements CompressionProvider

Field Summary

Fields

Modifier and Type	Field and Description
static int	IO_COMPRESSION_CODEC_SNAPPY_DEFAULT_BUFFERSIZE

Constructor Summary

Constructors

Constructor and Description
HadoopSnappyCompressionProvider()

Method Summary

Modifier and Type	Method and Description
HadoopSnappyCompressionInputStream	createInputStream(InputStream in) Creates an input stream that decompresses data (according to the provider's implementation)
HadoopSnappyCompressionOutputStream	createOutputStream(OutputStream out) Creates an output stream that compresses data (according to the provider's implementation)
static Object	getActiveSnappyShim() Locate the Snappy Shim for the active Hadoop Configuration via the Hadoop Configuration Util
String	getDefaultExtension() Gets the default file extension for this provider.
String	getDescription() Gets the name of this provider.
String	getName() Gets the name of this provider.
static boolean	isHadoopSnappyAvailable() Tests whether hadoop-snappy (not to be confused with other java-based snappy implementations such as jsnappy or snappy-java) plus the native snappy libraries are available.
boolean	supportsInput() Whether this compression provider supports input streams
boolean	supportsOutput() Whether this compression provider supports output streams

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

IO_COMPRESSION_CODEC_SNAPPY_DEFAULT_BUFFERSIZE

public static final int IO_COMPRESSION_CODEC_SNAPPY_DEFAULT_BUFFERSIZE

See Also:

Constant Field Values

Constructor Detail

HadoopSnappyCompressionProvider

public HadoopSnappyCompressionProvider()

Method Detail

getActiveSnappyShim

public static Object getActiveSnappyShim()
                                  throws Exception

Locate the Snappy Shim for the active Hadoop Configuration via the Hadoop Configuration Util

Returns:

A org.pentaho.hadoop.shim.spi.SnappyShim to interact with Snappy

Throws:

Exception - Error locating a valid Snappy shim:

• org.pentaho.hadoop.hive.jdbc.HadoopConfigurationUtil could not be located
• No active Hadoop configuration
• Active Hadoop configuration doesn't support Snappy

isHadoopSnappyAvailable

public static boolean isHadoopSnappyAvailable()

Tests whether hadoop-snappy (not to be confused with other java-based snappy implementations such as jsnappy or snappy-java) plus the native snappy libraries are available.

Returns:

true if hadoop-snappy is available on the classpath

createInputStream

public HadoopSnappyCompressionInputStream createInputStream(InputStream in)
                                                     throws IOException

Description copied from interface: CompressionProvider

Creates an input stream that decompresses data (according to the provider's implementation)

Specified by:

createInputStream in interface CompressionProvider

Parameters:

in - An existing input stream to wrap

Returns:

a CompressionInputStream object that decompresses incoming data

Throws:

IOException

supportsInput

public boolean supportsInput()

Description copied from interface: CompressionProvider

Whether this compression provider supports input streams

Specified by:

supportsInput in interface CompressionProvider

Returns:

true if the provider supports input streams, false otherwise

createOutputStream

public HadoopSnappyCompressionOutputStream createOutputStream(OutputStream out)
                                                       throws IOException

Description copied from interface: CompressionProvider

Creates an output stream that compresses data (according to the provider's implementation)

Specified by:

createOutputStream in interface CompressionProvider

Parameters:

out - An existing output stream to wrap

Returns:

a CompressionOutputStream object that compresses outgoing data

Throws:

IOException

supportsOutput

public boolean supportsOutput()

Description copied from interface: CompressionProvider

Whether this compression provider supports output streams

Specified by:

supportsOutput in interface CompressionProvider

Returns:

true if the provider supports output streams, false otherwise

getDescription

public String getDescription()

Description copied from interface: CompressionProvider

Gets the name of this provider. Used for display e.g.

Specified by:

getDescription in interface CompressionProvider

Returns:

A String containing a description of this provider

getName

public String getName()

Description copied from interface: CompressionProvider

Gets the name of this provider. Used for display and as a reference in saved artifacts (transformations, e.g.)

Specified by:

getName in interface CompressionProvider

Returns:

A String containing the name of this provider

getDefaultExtension

public String getDefaultExtension()

Description copied from interface: CompressionProvider

Gets the default file extension for this provider. If the streams are wrapped in File streams, this method can be used to determine an appropriate extension to append to the filename so the file will be recognized as an artifact of the compression mechanism (.zip, .bz2, e.g.)

Specified by:

getDefaultExtension in interface CompressionProvider

Returns:

A String containing the default file extension for this provider