org.pentaho.di.trans.steps.csvinput
Class CsvInputData

java.lang.Object
  org.pentaho.di.trans.step.BaseStepData
      org.pentaho.di.trans.steps.csvinput.CsvInputData

All Implemented Interfaces:

StepDataInterface

public class CsvInputData
extends BaseStepData
implements StepDataInterface

Since:

24-jan-2005

Author:

Matt

Nested Class Summary

Nested classes/interfaces inherited from class org.pentaho.di.trans.step.BaseStepData

BaseStepData.StepExecutionStatus

Field Summary

ByteBuffer	bb
byte[]	binaryFilename
long	blockToRead
int	bufferSize
byte[]	byteBuffer
long	bytesToSkipInFirstFile
RowMetaInterface	convertRowMeta
CrLfMatcherInterface	crLfMatcher
byte[]	delimiter
PatternMatcherInterface	delimiterMatcher
byte[]	enclosure
PatternMatcherInterface	enclosureMatcher
EncodingType	encodingType
int	endBuffer
long	endPosition
FileChannel	fc
int	filenameFieldIndex
String[]	filenames
int	filenr
long	fileSize
List<Long>	fileSizes
FileInputStream	fis
boolean	isAddingRowNumber
RowMetaInterface	outputRowMeta
boolean	parallel
int	preferredBufferSize
long	rowNumber
int	rownumFieldIndex
int	startBuffer
int	startFilenr
long	startPosition
int	stepNumber
boolean	stopReading
long	totalBytesRead
long	totalFileSize
int	totalNumberOfSteps

Constructor Summary

CsvInputData()

Method Summary

boolean	increaseEndBuffer() Increase the endBuffer pointer by one. If there is not enough room in the buffer to go there, resize the byte buffer and read more data. if there is no more data to read and if the endBuffer pointer has reached the end of the byte buffer, we return true.
int	readBufferFromFile()
byte[]	removeEscapedEnclosures(byte[] field, int nrEnclosuresFound) [abcd "" defg] --> [abcd " defg] [""""] --> [""] [""] --> ["]
void	resizeByteBufferArray()

Methods inherited from class org.pentaho.di.trans.step.BaseStepData

getStatus, isDisposed, isEmpty, isFinished, isIdle, isInitialising, isRunning, isStopped, setStatus

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.pentaho.di.trans.step.StepDataInterface

getStatus, isDisposed, isEmpty, isFinished, isIdle, isInitialising, isRunning, setStatus

Field Detail

fc

public FileChannel fc

bb

public ByteBuffer bb

convertRowMeta

public RowMetaInterface convertRowMeta

outputRowMeta

public RowMetaInterface outputRowMeta

byteBuffer

public byte[] byteBuffer

startBuffer

public int startBuffer

endBuffer

public int endBuffer

bufferSize

public int bufferSize

delimiter

public byte[] delimiter

enclosure

public byte[] enclosure

preferredBufferSize

public int preferredBufferSize

filenames

public String[] filenames

filenr

public int filenr

startFilenr

public int startFilenr

binaryFilename

public byte[] binaryFilename

fileSize

public long fileSize

fis

public FileInputStream fis

isAddingRowNumber

public boolean isAddingRowNumber

rowNumber

public long rowNumber

stopReading

public boolean stopReading

stepNumber

public int stepNumber

totalNumberOfSteps

public int totalNumberOfSteps

fileSizes

public List<Long> fileSizes

totalFileSize

public long totalFileSize

blockToRead

public long blockToRead

startPosition

public long startPosition

endPosition

public long endPosition

bytesToSkipInFirstFile

public long bytesToSkipInFirstFile

totalBytesRead

public long totalBytesRead

parallel

public boolean parallel

filenameFieldIndex

public int filenameFieldIndex

rownumFieldIndex

public int rownumFieldIndex

encodingType

public EncodingType encodingType

delimiterMatcher

public PatternMatcherInterface delimiterMatcher

enclosureMatcher

public PatternMatcherInterface enclosureMatcher

crLfMatcher

public CrLfMatcherInterface crLfMatcher

Constructor Detail

CsvInputData

public CsvInputData()

Method Detail

resizeByteBufferArray

public void resizeByteBufferArray()

readBufferFromFile

public int readBufferFromFile()
                       throws IOException

Throws:

IOException

increaseEndBuffer

public boolean increaseEndBuffer()
                          throws IOException

Increase the endBuffer pointer by one.
If there is not enough room in the buffer to go there, resize the byte buffer and read more data.
if there is no more data to read and if the endBuffer pointer has reached the end of the byte buffer, we return true.

Returns:

true if we reached the end of the byte buffer.

Throws:

IOException - In case we get an error reading from the input file.

removeEscapedEnclosures

public byte[] removeEscapedEnclosures(byte[] field,
                                      int nrEnclosuresFound)

    
      [abcd "" defg] --> [abcd " defg]
      [""""] --> [""]
      [""] --> ["]
      

Returns:

the byte array with escaped enclosures escaped.