org.pentaho.di.trans.steps.csvinput
Class CsvInputData

java.lang.Object
  extended by org.pentaho.di.trans.step.BaseStepData
      extended by org.pentaho.di.trans.steps.csvinput.CsvInputData
All Implemented Interfaces:
StepDataInterface

public class CsvInputData
extends BaseStepData
implements StepDataInterface

Since:
24-jan-2005
Author:
Matt

Nested Class Summary
 
Nested classes/interfaces inherited from class org.pentaho.di.trans.step.BaseStepData
BaseStepData.StepExecutionStatus
 
Field Summary
 ByteBuffer bb
           
 byte[] binaryFilename
           
 long blockToRead
           
 int bufferSize
           
 byte[] byteBuffer
           
 long bytesToSkipInFirstFile
           
 RowMetaInterface convertRowMeta
           
 CrLfMatcherInterface crLfMatcher
           
 byte[] delimiter
           
 PatternMatcherInterface delimiterMatcher
           
 byte[] enclosure
           
 PatternMatcherInterface enclosureMatcher
           
 EncodingType encodingType
           
 int endBuffer
           
 long endPosition
           
 FileChannel fc
           
 int filenameFieldIndex
           
 String[] filenames
           
 int filenr
           
 long fileSize
           
 List<Long> fileSizes
           
 FileInputStream fis
           
 boolean isAddingRowNumber
           
 RowMetaInterface outputRowMeta
           
 boolean parallel
           
 int preferredBufferSize
           
 long rowNumber
           
 int rownumFieldIndex
           
 int startBuffer
           
 int startFilenr
           
 long startPosition
           
 int stepNumber
           
 boolean stopReading
           
 long totalBytesRead
           
 long totalFileSize
           
 int totalNumberOfSteps
           
 
Constructor Summary
CsvInputData()
           
 
Method Summary
 boolean increaseEndBuffer()
          Increase the endBuffer pointer by one.
If there is not enough room in the buffer to go there, resize the byte buffer and read more data.
if there is no more data to read and if the endBuffer pointer has reached the end of the byte buffer, we return true.
 int readBufferFromFile()
           
 byte[] removeEscapedEnclosures(byte[] field, int nrEnclosuresFound)
           [abcd "" defg] --> [abcd " defg] [""""] --> [""] [""] --> ["]
 void resizeByteBufferArray()
           
 
Methods inherited from class org.pentaho.di.trans.step.BaseStepData
getStatus, isDisposed, isEmpty, isFinished, isIdle, isInitialising, isRunning, isStopped, setStatus
 
Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 
Methods inherited from interface org.pentaho.di.trans.step.StepDataInterface
getStatus, isDisposed, isEmpty, isFinished, isIdle, isInitialising, isRunning, setStatus
 

Field Detail

fc

public FileChannel fc

bb

public ByteBuffer bb

convertRowMeta

public RowMetaInterface convertRowMeta

outputRowMeta

public RowMetaInterface outputRowMeta

byteBuffer

public byte[] byteBuffer

startBuffer

public int startBuffer

endBuffer

public int endBuffer

bufferSize

public int bufferSize

delimiter

public byte[] delimiter

enclosure

public byte[] enclosure

preferredBufferSize

public int preferredBufferSize

filenames

public String[] filenames

filenr

public int filenr

startFilenr

public int startFilenr

binaryFilename

public byte[] binaryFilename

fileSize

public long fileSize

fis

public FileInputStream fis

isAddingRowNumber

public boolean isAddingRowNumber

rowNumber

public long rowNumber

stopReading

public boolean stopReading

stepNumber

public int stepNumber

totalNumberOfSteps

public int totalNumberOfSteps

fileSizes

public List<Long> fileSizes

totalFileSize

public long totalFileSize

blockToRead

public long blockToRead

startPosition

public long startPosition

endPosition

public long endPosition

bytesToSkipInFirstFile

public long bytesToSkipInFirstFile

totalBytesRead

public long totalBytesRead

parallel

public boolean parallel

filenameFieldIndex

public int filenameFieldIndex

rownumFieldIndex

public int rownumFieldIndex

encodingType

public EncodingType encodingType

delimiterMatcher

public PatternMatcherInterface delimiterMatcher

enclosureMatcher

public PatternMatcherInterface enclosureMatcher

crLfMatcher

public CrLfMatcherInterface crLfMatcher
Constructor Detail

CsvInputData

public CsvInputData()
Method Detail

resizeByteBufferArray

public void resizeByteBufferArray()

readBufferFromFile

public int readBufferFromFile()
                       throws IOException
Throws:
IOException

increaseEndBuffer

public boolean increaseEndBuffer()
                          throws IOException
Increase the endBuffer pointer by one.
If there is not enough room in the buffer to go there, resize the byte buffer and read more data.
if there is no more data to read and if the endBuffer pointer has reached the end of the byte buffer, we return true.

Returns:
true if we reached the end of the byte buffer.
Throws:
IOException - In case we get an error reading from the input file.

removeEscapedEnclosures

public byte[] removeEscapedEnclosures(byte[] field,
                                      int nrEnclosuresFound)
    
      [abcd "" defg] --> [abcd " defg]
      [""""] --> [""]
      [""] --> ["]
      

Returns:
the byte array with escaped enclosures escaped.