org.pentaho.hadoop

Class HadoopCompression

java.lang.Object

org.pentaho.hadoop.HadoopCompression

public class HadoopCompression
extends Object

Helper class for determining (via reflection) whether various Hadoop compression codecs (such as Snappy) are available on the classpath and for returning Input/Output streams for reading/writing.

Author:

Mark Hall (mhall{[at]}pentaho{[dot]}com)

Field Summary

Fields

Modifier and Type	Field and Description
static int	IO_COMPRESSION_CODEC_SNAPPY_DEFAULT_BUFFERSIZE

Constructor Summary

Constructors

Constructor and Description
HadoopCompression()

Method Summary

Methods

Modifier and Type	Method and Description
static InputStream	getSnappyInputStream(InputStream in) Gets a CompressionInputStream that uses the snappy codec and wraps the supplied base input stream.
static InputStream	getSnappyInputStream(int bufferSize, InputStream in) Gets an InputStream that uses the snappy codec and wraps the supplied base input stream.
static OutputStream	getSnappyOutputStream(int bufferSize, OutputStream out) Gets an OutputStream that uses the snappy codec and wraps the supplied base output stream.
OutputStream	getSnappyOutputStream(OutputStream out) Gets an OutputStream that uses the snappy codec and wraps the supplied base output stream.
static boolean	isHadoopSnappyAvailable() Tests whether hadoop-snappy (not to be confused with other java-based snappy implementations such as jsnappy or snappy-java) plus the native snappy libraries are available.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

IO_COMPRESSION_CODEC_SNAPPY_DEFAULT_BUFFERSIZE

public static final int IO_COMPRESSION_CODEC_SNAPPY_DEFAULT_BUFFERSIZE

See Also:

Constant Field Values

Constructor Detail

HadoopCompression

public HadoopCompression()

Method Detail

isHadoopSnappyAvailable

public static boolean isHadoopSnappyAvailable()

Tests whether hadoop-snappy (not to be confused with other java-based snappy implementations such as jsnappy or snappy-java) plus the native snappy libraries are available.

Returns:

true if hadoop-snappy is available on the classpath

getSnappyInputStream

public static InputStream getSnappyInputStream(InputStream in)
                                        throws Exception

Gets a CompressionInputStream that uses the snappy codec and wraps the supplied base input stream.

Parameters:

in - the base input stream to wrap around

Returns:

an InputStream that uses the Snappy codec

Throws:

Exception - if snappy is not available or an error occurs during reflection

getSnappyInputStream

public static InputStream getSnappyInputStream(int bufferSize,
                               InputStream in)
                                        throws Exception

Gets an InputStream that uses the snappy codec and wraps the supplied base input stream.

Parameters:

the - buffer size for the codec to use (in bytes)

in - the base input stream to wrap around

Returns:

an InputStream that uses the Snappy codec

Throws:

Exception - if snappy is not available or an error occurs during reflection

getSnappyOutputStream

public OutputStream getSnappyOutputStream(OutputStream out)
                                   throws Exception

Gets an OutputStream that uses the snappy codec and wraps the supplied base output stream.

Parameters:

the - buffer size for the codec to use (in bytes)

out - the base output stream to wrap around

Returns:

a OutputStream that uses the Snappy codec

Throws:

Exception - if snappy is not available or an error occurs during reflection

getSnappyOutputStream

public static OutputStream getSnappyOutputStream(int bufferSize,
                                 OutputStream out)
                                          throws Exception

Gets an OutputStream that uses the snappy codec and wraps the supplied base output stream.

Parameters:

the - buffer size for the codec to use (in bytes)

out - the base output stream to wrap around

Returns:

a OutputStream that uses the Snappy codec

Throws:

Exception - if snappy is not available or an error occurs during reflection