org.pentaho.di.trans.steps.fileinput

Class BaseFileInputStep<M extends BaseFileInputStepMeta,D extends BaseFileInputStepData>

java.lang.Object

org.pentaho.di.trans.step.BaseStep

org.pentaho.di.trans.steps.fileinput.BaseFileInputStep<M,D>

All Implemented Interfaces:

org.pentaho.di.core.ExtensionDataInterface, HasLogChannelInterface, org.pentaho.di.core.logging.LoggingObjectInterface, org.pentaho.di.core.variables.VariableSpace, StepInterface, IBaseFileInputStepControl

Direct Known Subclasses:

TextFileInput

public abstract class BaseFileInputStep<M extends BaseFileInputStepMeta,D extends BaseFileInputStepData>
extends BaseStep
implements IBaseFileInputStepControl

This class contains base functionality for file-based input steps.

Author:

Alexander Buloichik

Field Summary

Fields inherited from class org.pentaho.di.trans.step.BaseStep

first, linesInput, linesOutput, linesRead, linesRejected, linesSkipped, linesUpdated, linesWritten, terminator, terminator_rows

Constructor Summary

Constructors

Constructor and Description
BaseFileInputStep(StepMeta stepMeta, StepDataInterface stepDataInterface, int copyNr, TransMeta transMeta, Trans trans)

Method Summary

Methods

Modifier and Type	Method and Description
boolean	checkFeedback(long lines) Check feedback.
void	dispose(StepMetaInterface smi, StepDataInterface sdi) Dispose step.
boolean	failAfterBadFile(String errorMsg)
boolean	init(StepMetaInterface smi, StepDataInterface sdi) Initialize step before execute.
boolean	processRow(StepMetaInterface smi, StepDataInterface sdi) Process next row.

Methods inherited from class org.pentaho.di.trans.step.BaseStep

addResultFile, addRowListener, addStepListener, batchComplete, buildLog, canProcessOneRow, cleanup, closeQuietly, copyVariablesFrom, decrementLinesRead, decrementLinesWritten, dispatch, environmentSubstitute, environmentSubstitute, fieldSubstitute, findInputRowSet, findInputRowSet, findOutputRowSet, findOutputRowSet, getBooleanValueOfVariable, getClusterSize, getContainerObjectId, getCopy, getCurrentInputRowSetNr, getCurrentOutputRowSetNr, getDispatcher, getErrorRowMeta, getErrors, getExtensionDataMap, getFilename, getInputRowMeta, getInputRowSets, getLinesInput, getLinesOutput, getLinesRead, getLinesRejected, getLinesSkipped, getLinesUpdated, getLinesWritten, getLogChannel, getLogChannelId, getLogFields, getLogLevel, getMetaStore, getNextClassNr, getObjectCopy, getObjectId, getObjectName, getObjectRevision, getObjectType, getOutputRowSets, getParent, getParentVariableSpace, getPartitionID, getPartitionTargets, getPreviewRowMeta, getProcessed, getRegistrationDate, getRemoteInputSteps, getRemoteOutputSteps, getRepartitioning, getRepository, getRepositoryDirectory, getResultFiles, getRow, getRowFrom, getRowListeners, getRuntime, getServerSockets, getSlaveNr, getSocketRepository, getStatus, getStatusDescription, getStepDataInterface, getStepID, getStepListeners, getStepMeta, getStepMetaInterface, getStepname, getTrans, getTransMeta, getTypeId, getUniqueStepCountAcrossSlaves, getUniqueStepNrAcrossSlaves, getVariable, getVariable, identifyErrorOutput, incrementLinesInput, incrementLinesOutput, incrementLinesRead, incrementLinesRejected, incrementLinesSkipped, incrementLinesUpdated, incrementLinesWritten, initBeforeStart, initializeVariablesFrom, injectVariables, isBasic, isDebug, isDetailed, isDistributed, isForcingSeparateLogging, isGatheringMetrics, isInitialising, isMapping, isPartitioned, isPaused, isRowLevel, isRunning, isStopped, isUsingThreadPriorityManagment, listVariables, logBasic, logBasic, logDebug, logDebug, logDetailed, logDetailed, logError, logError, logError, logMinimal, logMinimal, logRowlevel, logRowlevel, logSummary, markStart, markStop, outputIsDone, pauseRunning, putError, putRow, putRowTo, removeRowListener, resumeRunning, rowsetInputSize, rowsetOutputSize, safeModeChecking, setCarteObjectId, setCopy, setCurrentInputRowSetNr, setCurrentOutputRowSetNr, setDistributed, setErrorRowMeta, setErrors, setForcingSeparateLogging, setGatheringMetrics, setInputRowMeta, setInputRowSets, setInternalVariables, setLinesInput, setLinesOutput, setLinesRead, setLinesRejected, setLinesSkipped, setLinesUpdated, setLinesWritten, setLogLevel, setMetaStore, setOutputDone, setOutputRowSets, setParentVariableSpace, setPartitioned, setPartitionID, setPartitionTargets, setPaused, setPaused, setPreviewRowMeta, setRepartitioning, setRepository, setRunning, setServerSockets, setSocketRepository, setStepDataInterface, setStepListeners, setStepMeta, setStepMetaInterface, setStepname, setStopped, setTransMeta, setUsingThreadPriorityManagment, setVariable, shareVariablesWith, stopAll, stopRunning, stopRunning, toString

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.pentaho.di.trans.steps.fileinput.IBaseFileInputStepControl

getErrors, getLinesInput, getLinesWritten, incrementLinesInput, incrementLinesUpdated, putRow, setErrors, stopAll

Constructor Detail

BaseFileInputStep

public BaseFileInputStep(StepMeta stepMeta,
                 StepDataInterface stepDataInterface,
                 int copyNr,
                 TransMeta transMeta,
                 Trans trans)

Method Detail

init

public boolean init(StepMetaInterface smi,
           StepDataInterface sdi)

Initialize step before execute.

Specified by:

init in interface StepInterface

Overrides:

init in class BaseStep

Parameters:

smi - The metadata to work with

sdi - The data to initialize

processRow

public boolean processRow(StepMetaInterface smi,
                 StepDataInterface sdi)
                   throws org.pentaho.di.core.exception.KettleException

Process next row. This methods opens next file automatically.

Specified by:

processRow in interface StepInterface

Overrides:

processRow in class BaseStep

Parameters:

smi - The steps metadata to work with

sdi - The steps temporary working data to work with (database connections, result sets, caches, temporary variables, etc.)

Returns:

false if no more rows can be processed or an error occurred.

Throws:

org.pentaho.di.core.exception.KettleException

checkFeedback

public boolean checkFeedback(long lines)

Description copied from class: BaseStep

Check feedback.

Specified by:

checkFeedback in interface IBaseFileInputStepControl

Parameters:

lines - the lines

Returns:

true, if successful

dispose

public void dispose(StepMetaInterface smi,
           StepDataInterface sdi)

Dispose step.

Specified by:

dispose in interface StepInterface

Overrides:

dispose in class BaseStep

Parameters:

smi - The metadata to work with

sdi - The data to dispose of

failAfterBadFile

public boolean failAfterBadFile(String errorMsg)

Specified by:

failAfterBadFile in interface IBaseFileInputStepControl

Parameters:

errorMsg - Message to send to rejected row if enabled

Returns:

If should stop processing after having problems with a file