BaseStep

Overview

Package

Class

Use

Tree

Deprecated

Index

Help

PREV CLASS NEXT CLASS

FRAMES NO FRAMES

SUMMARY: NESTED | FIELD | CONSTR | METHOD

DETAIL: FIELD | CONSTR | METHOD

org.pentaho.di.trans.step
Class BaseStep

java.lang.Object
  org.pentaho.di.trans.step.BaseStep

All Implemented Interfaces:: HasLogChannelInterface, LoggingObjectInterface, VariableSpace, StepInterface

Direct Known Subclasses:: Abort, AbstractStep, AccessInput, AccessOutput, AddSequence, AddXML, AggregateRows, AnalyticQuery, Append, AutoDoc, BlockingStep, BlockUntilStepsFinish, Calculator, ChangeFileEncoding, CheckSum, CloneRow, ClosureGenerator, ColumnExists, CombinationLookup, Constant, CreditCardValidator, CsvInput, CubeInput, CubeOutput, DatabaseJoin, DatabaseLookup, DataGrid, DBProc, Delay, Delete, Denormaliser, DetectEmptyStream, DetectLastRow, DimensionLookup, DummyTrans, DynamicSQLRow, ElasticSearchBulk, ExcelInput, ExcelOutput, ExcelWriterStep, ExecProcess, ExecSQL, ExecSQLRow, FieldsChangeSequence, FieldSplitter, FileExists, FileLocked, FilesFromResult, FilesToResult, FilterRows, FixedInput, Flattener, Formula, FuzzyMatch, GaInputStep, GetFileNames, GetFilesRowsCount, GetPreviousRowField, GetRepositoryNames, GetSlaveSequence, GetSubFolders, GetTableNames, GetVariable, GetXMLData, GPBulkLoader, GroupBy, HTTP, HTTPPOST, IfNull, InfobrightLoader, IngresVectorwiseLoader, Injector, InsertUpdate, Janino, JavaFilter, JoinRows, JsonInput, JsonOutput, LDAPInput, LDAPOutput, LDIFInput, LoadFileInput, LucidDBBulkLoader, LucidDBStreamingLoader, Mail, MailInput, MailValidator, Mapping, MappingInput, MappingOutput, MemoryGroupBy, MergeJoin, MergeRows, MetaInject, MondrianInput, MonetDBBulkLoader, MultiMergeJoin, MySQLBulkLoader, Normaliser, NullIf, NumberRange, OlapInput, OraBulkLoader, ParGzipCsvInput, PentahoReportingOutput, PGBulkLoader, PrioritizeStreams, ProcessFiles, PropertyInput, PropertyOutput, RandomCCNumberGenerator, RandomValue, RegexEval, ReplaceString, ReservoirSampling, Rest, RowGenerator, RowsFromResult, RowsToResult, RssInput, RssOutput, Rules, SalesforceDelete, SalesforceInput, SalesforceInsert, SalesforceUpdate, SalesforceUpsert, SampleRows, SapInput, Script, ScriptValuesMod, SecretKeyGenerator, SelectValues, SetValueConstant, SetValueField, SetVariable, SingleThreader, SocketReader, SocketWriter, SortedMerge, SortRows, SplitFieldToRows, SQLFileOutput, SSH, StepMetastructure, StepsMetrics, StreamLookup, StringCut, StringOperations, SwitchCase, SymmetricCryptoTrans, SynchronizeAfterMerge, SyslogMessage, SystemData, TableExists, TableInput, TableOutput, TextFileInput, TextFileOutput, UniqueRows, UniqueRowsByHashSet, UnivariateStats, Update, UserDefinedJavaClass, Validator, ValueMapper, WebService, WebServiceAvailable, WriteToLog, XBaseInput, XMLInput, XMLInputSax, XMLInputStream, XMLJoin, XMLOutput, XsdValidator, Xslt, YamlInput

public class BaseStep
extends Object
implements VariableSpace, StepInterface, LoggingObjectInterface
extends Object
implements VariableSpace, StepInterface, LoggingObjectInterface

Field Summary
`boolean`	`first`
`long`	`linesInput` Deprecated. please use the supplied getters, setters and increment/decrement methods
`long`	`linesOutput` Deprecated. please use the supplied getters, setters and increment/decrement methods
`long`	`linesRead` Deprecated. please use the supplied getters, setters and increment/decrement methods
`long`	`linesRejected` Deprecated. please use the supplied getters, setters and increment/decrement methods
`long`	`linesSkipped` Deprecated. please use the supplied getters, setters and increment/decrement methods
`long`	`linesUpdated` Deprecated. please use the supplied getters, setters and increment/decrement methods
`long`	`linesWritten` Deprecated. please use the supplied getters, setters and increment/decrement methods
`boolean`	`terminator`
`List<Object[]>`	`terminator_rows`

Constructor Summary
`BaseStep(StepMeta stepMeta, StepDataInterface stepDataInterface, int copyNr, TransMeta transMeta, Trans trans)` This is the base step that forms that basis for all steps.

Method Summary
`void`	`addResultFile(ResultFile resultFile)`
`void`	`addRowListener(RowListener rowListener)` Add a rowlistener to the step allowing you to inspect (or manipulate, be careful) the rows coming in or exiting the step.
`void`	`addStepListener(StepListener stepListener)` Attach a step listener to be notified when a step arrives in a certain state.
`void`	`batchComplete()` Calling this method will alert the step that we finished passing a batch of records to the step.
`RowMetaAndData`	`buildLog(String sname, int copynr, long lines_read, long lines_written, long lines_updated, long lines_skipped, long errors, Date start_date, Date end_date)`
`boolean`	`canProcessOneRow()` This method checks if the step is capable of processing at least one row.
`void`	`cleanup()` Call this method typically, after ALL the slave transformations in a clustered run have finished.
`static void`	`closeQuietly(Closeable cl)`
`void`	`copyVariablesFrom(VariableSpace space)` Copy the variables from another space, without initializing with the defaults.
`long`	`decrementLinesRead()` Decrements the number of lines read from previous steps by one
`long`	`decrementLinesWritten()` Decrements the number of lines written to next steps by one
`void`	`dispatch()` This method finds the surrounding steps and rowsets for this base step.
`void`	`dispose(StepMetaInterface smi, StepDataInterface sdi)` Dispose of this step: close files, empty logs, etc.
`String`	`environmentSubstitute(String aString)` Substitute the string using the current variable space.
`String[]`	`environmentSubstitute(String[] aString)` Replaces environment variables in an array of strings.
`RowSet`	`findInputRowSet(String sourceStep)`
`RowSet`	`findInputRowSet(String from, int fromcopy, String to, int tocopy)`
`RowSet`	`findOutputRowSet(String targetStep)`
`RowSet`	`findOutputRowSet(String from, int fromcopy, String to, int tocopy)` Find an output rowset in a running transformation.
`boolean`	`getBooleanValueOfVariable(String variableName, boolean defaultValue)` This method returns a boolean for the new variable check boxes.
`int`	`getClusterSize()`
`String`	`getContainerObjectId()`
`int`	`getCopy()`
`Trans`	`getDispatcher()`
`RowMetaInterface`	`getErrorRowMeta()`
`long`	`getErrors()` Get the number of errors
`String`	`getFilename()`
`RowMetaInterface`	`getInputRowMeta()`
`List<RowSet>`	`getInputRowSets()`
`long`	`getLinesInput()`
`long`	`getLinesOutput()`
`long`	`getLinesRead()`
`long`	`getLinesRejected()`
`long`	`getLinesSkipped()`
`long`	`getLinesUpdated()`
`long`	`getLinesWritten()`
`LogChannelInterface`	`getLogChannel()`
`String`	`getLogChannelId()`
`static RowMetaInterface`	`getLogFields(String comm)`
`LogLevel`	`getLogLevel()`
`int`	`getNextClassNr()`
`String`	`getObjectCopy()`
`ObjectId`	`getObjectId()`
`String`	`getObjectName()`
`ObjectRevision`	`getObjectRevision()`
`LoggingObjectType`	`getObjectType()`
`List<RowSet>`	`getOutputRowSets()`
`LoggingObjectInterface`	`getParent()`
`VariableSpace`	`getParentVariableSpace()` Get the parent of the variable space.
`String`	`getPartitionID()`
`Map<String,BlockingRowSet>`	`getPartitionTargets()`
`RowMetaInterface`	`getPreviewRowMeta()`
`long`	`getProcessed()`
`Date`	`getRegistrationDate()` Stub
`List<RemoteStep>`	`getRemoteInputSteps()`
`List<RemoteStep>`	`getRemoteOutputSteps()`
`int`	`getRepartitioning()`
`RepositoryDirectory`	`getRepositoryDirectory()`
`Map<String,ResultFile>`	`getResultFiles()`
`Object[]`	`getRow()` In case of getRow, we receive data from previous steps through the input rowset.
`Object[]`	`getRowFrom(RowSet rowSet)`
`List<RowListener>`	`getRowListeners()`
`long`	`getRuntime()`
`List<ServerSocket>`	`getServerSockets()`
`int`	`getSlaveNr()`
`SocketRepository`	`getSocketRepository()`
`BaseStepData.StepExecutionStatus`	`getStatus()`
`String`	`getStatusDescription()`
`StepDataInterface`	`getStepDataInterface()`
`String`	`getStepID()`
`List<StepListener>`	`getStepListeners()`
`StepMeta`	`getStepMeta()`
`StepMetaInterface`	`getStepMetaInterface()`
`String`	`getStepname()` Get the name of the step.
`Trans`	`getTrans()`
`TransMeta`	`getTransMeta()`
`String`	`getTypeId()` Support for CheckResultSourceInterface
`int`	`getUniqueStepCountAcrossSlaves()`
`int`	`getUniqueStepNrAcrossSlaves()`
`String`	`getVariable(String variableName)` Get the value of a variable.
`String`	`getVariable(String variableName, String defaultValue)` Get the value of a variable with a default in case the variable is not found.
`void`	`identifyErrorOutput()` To be used to flag an error output channel of a step prior to execution for performance reasons.
`long`	`incrementLinesInput()` Increments the number of lines read from an input source: database, file, socket, etc.
`long`	`incrementLinesOutput()` Increments the number of lines written to an output target: database, file, socket, etc.
`long`	`incrementLinesRead()` Increments the number of lines read from previous steps by one
`long`	`incrementLinesRejected()` Increments the number of lines rejected to an error handling step
`long`	`incrementLinesSkipped()` Increments the number of lines skipped
`long`	`incrementLinesUpdated()` Increments the number of lines updated in an output target: database, file, socket, etc.
`long`	`incrementLinesWritten()` Increments the number of lines written to next steps by one
`boolean`	`init(StepMetaInterface smi, StepDataInterface sdi)` Initialize and do work where other steps need to wait for...
`void`	`initBeforeStart()` This method is executed by Trans right before the threads start and right after initialization.
`void`	`initializeVariablesFrom(VariableSpace parent)` Initialize variable space using the defaults, copy over the variables from the parent (using copyVariablesFrom()), after this the "injected" variables should be inserted (injectVariables()).
`void`	`injectVariables(Map<String,String> prop)` Inject variables.
`boolean`	`isBasic()`
`boolean`	`isDebug()`
`boolean`	`isDetailed()`
`boolean`	`isDistributed()`
`boolean`	`isInitialising()`
`boolean`	`isMapping()`
`boolean`	`isPartitioned()`
`boolean`	`isPaused()`
`boolean`	`isRowLevel()`
`boolean`	`isRunning()`
`boolean`	`isStopped()`
`boolean`	`isUsingThreadPriorityManagment()`
`String[]`	`listVariables()` List the variables (not the values) that are currently in the variable space.
`void`	`logBasic(String message)`
`void`	`logBasic(String message, Object... arguments)`
`void`	`logDebug(String message)`
`void`	`logDebug(String message, Object... arguments)`
`void`	`logDetailed(String message)`
`void`	`logDetailed(String message, Object... arguments)`
`void`	`logError(String message)`
`void`	`logError(String message, Object... arguments)`
`void`	`logError(String message, Throwable e)`
`void`	`logMinimal(String message)`
`void`	`logMinimal(String message, Object... arguments)`
`void`	`logRowlevel(String message)`
`void`	`logRowlevel(String message, Object... arguments)`
`void`	`logSummary()`
`void`	`markStart()` Mark the start time of the step.
`void`	`markStop()` Mark the end time of the step.
`boolean`	`outputIsDone()`
`void`	`pauseRunning()` Pause a running step
`boolean`	`processRow(StepMetaInterface smi, StepDataInterface sdi)` Perform the equivalent of processing one row.
`void`	`putError(RowMetaInterface rowMeta, Object[] row, long nrErrors, String errorDescriptions, String fieldNames, String errorCodes)`
`void`	`putRow(RowMetaInterface rowMeta, Object[] row)` putRow is used to copy a row, to the alternate rowset(s) This should get priority over everything else! (synchronized) If distribute is true, a row is copied only once to the output rowsets, otherwise copies are sent to each rowset!
`void`	`putRowTo(RowMetaInterface rowMeta, Object[] row, RowSet rowSet)` putRowTo is used to put a row in a certain specific RowSet.
`void`	`removeRowListener(RowListener rowListener)` Remove a rowlistener from this step.
`void`	`resumeRunning()` Resume a running step
`int`	`rowsetInputSize()`
`int`	`rowsetOutputSize()`
`static void`	`safeModeChecking(RowMetaInterface referenceRowMeta, RowMetaInterface rowMeta)`
`void`	`setCarteObjectId(String containerObjectId)`
`void`	`setCopy(int cop)`
`void`	`setDistributed(boolean distributed)`
`void`	`setErrorRowMeta(RowMetaInterface errorRowMeta)`
`void`	`setErrors(long e)` Sets the number of errors
`void`	`setInputRowMeta(RowMetaInterface rowMeta)`
`void`	`setInputRowSets(List<RowSet> inputRowSets)`
`void`	`setInternalVariables()`
`void`	`setLinesInput(long newLinesInputValue)`
`void`	`setLinesOutput(long newLinesOutputValue)`
`void`	`setLinesRead(long newLinesReadValue)`
`void`	`setLinesRejected(long newLinesRejectedValue)`
`void`	`setLinesSkipped(long newLinesSkippedValue)`
`void`	`setLinesUpdated(long newLinesUpdatedValue)`
`void`	`setLinesWritten(long newLinesWrittenValue)`
`void`	`setLogLevel(LogLevel logLevel)`
`void`	`setOutputDone()` Signal output done to destination steps
`void`	`setOutputRowSets(List<RowSet> outputRowSets)`
`void`	`setParentVariableSpace(VariableSpace parent)` Set the parent variable space
`void`	`setPartitioned(boolean partitioned)`
`void`	`setPartitionID(String partitionID)`
`void`	`setPartitionTargets(Map<String,BlockingRowSet> partitionTargets)`
`void`	`setPaused(AtomicBoolean paused)`
`void`	`setPaused(boolean paused)`
`void`	`setPreviewRowMeta(RowMetaInterface previewRowMeta)`
`void`	`setRepartitioning(int repartitioning)`
`void`	`setRunning(boolean running)` Flag the step as running or not
`void`	`setServerSockets(List<ServerSocket> serverSockets)`
`void`	`setSocketRepository(SocketRepository socketRepository)`
`void`	`setStepDataInterface(StepDataInterface stepDataInterface)`
`void`	`setStepListeners(List<StepListener> stepListeners)`
`void`	`setStepMeta(StepMeta stepMeta)`
`void`	`setStepMetaInterface(StepMetaInterface stepMetaInterface)`
`void`	`setStepname(String stepname)`
`void`	`setStopped(boolean stopped)`
`void`	`setTransMeta(TransMeta transMeta)`
`void`	`setUsingThreadPriorityManagment(boolean usingThreadPriorityManagment)`
`void`	`setVariable(String variableName, String variableValue)` Sets a variable in the Kettle Variables list.
`void`	`shareVariablesWith(VariableSpace space)` Share a variable space from another variable space.
`void`	`stopAll()` Flags all rowsets as stopped/completed/finished.
`void`	`stopRunning()` Deprecated.
`void`	`stopRunning(StepMetaInterface stepMetaInterface, StepDataInterface stepDataInterface)` Perform actions to stop a running step.
`String`	`toString()`

Methods inherited from class java.lang.Object
`equals, getClass, hashCode, notify, notifyAll, wait, wait, wait`

Field Detail

linesRead

public long linesRead

Deprecated. please use the supplied getters, setters and increment/decrement methods

nr of lines read from previous step(s)

linesWritten

public long linesWritten

Deprecated. please use the supplied getters, setters and increment/decrement methods

nr of lines written to next step(s)

linesInput

public long linesInput

Deprecated. please use the supplied getters, setters and increment/decrement methods

nr of lines read from file or database

linesOutput

public long linesOutput

Deprecated. please use the supplied getters, setters and increment/decrement methods

nr of lines written to file or database

linesUpdated

public long linesUpdated

Deprecated. please use the supplied getters, setters and increment/decrement methods

nr of updates in a database table or file

linesSkipped

public long linesSkipped

Deprecated. please use the supplied getters, setters and increment/decrement methods

nr of lines skipped

linesRejected

public long linesRejected

Deprecated. please use the supplied getters, setters and increment/decrement methods

total sleep time in ns caused by an empty input buffer (previous step is slow)

first

public boolean first

terminator

public boolean terminator

terminator_rows

public List<Object[]> terminator_rows

Constructor Detail

BaseStep

public BaseStep(StepMeta stepMeta,
                StepDataInterface stepDataInterface,
                int copyNr,
                TransMeta transMeta,
                Trans trans)

This is the base step that forms that basis for all steps. You can derive from this class to implement your own steps.

Parameters:: stepMeta - The StepMeta object to run.; stepDataInterface - the data object to store temporary data, database connections, caches, result sets, hashtables etc.; copyNr - The copynumber for this step.; transMeta - The TransInfo of which the step stepMeta is part of.; trans - The (running) transformation to obtain information shared among the steps.

Method Detail

init

public boolean init(StepMetaInterface smi,
                    StepDataInterface sdi)

Description copied from interface: StepInterface

Initialize and do work where other steps need to wait for...

Specified by:: init in interface StepInterface

Parameters:: smi - The metadata to work with; sdi - The data to initialize

dispose

public void dispose(StepMetaInterface smi,
                    StepDataInterface sdi)

Description copied from interface: StepInterface

Dispose of this step: close files, empty logs, etc.

Specified by:: dispose in interface StepInterface

Parameters:: smi - The metadata to work with; sdi - The data to dispose of

cleanup

public void cleanup()

Description copied from interface: StepInterface

Call this method typically, after ALL the slave transformations in a clustered run have finished.

Specified by:: cleanup in interface StepInterface

getProcessed

public long getProcessed()

Specified by:: getProcessed in interface StepInterface

Returns:: The number of "processed" lines of a step. Well, a representable metric for that anyway.

setCopy

public void setCopy(int cop)

getCopy

public int getCopy()

Specified by:: getCopy in interface StepInterface

Returns:: The steps copy number (default 0)

getErrors

public long getErrors()

Description copied from interface: StepInterface

Get the number of errors

Specified by:: getErrors in interface StepInterface

Returns:: the number of errors

setErrors

public void setErrors(long e)

Description copied from interface: StepInterface

Sets the number of errors

Specified by:: setErrors in interface StepInterface

Parameters:: e - the number of errors to set

getLinesRead

public long getLinesRead()

Specified by:: getLinesRead in interface StepInterface

Returns:: Returns the number of lines read from previous steps

incrementLinesRead

public long incrementLinesRead()

Increments the number of lines read from previous steps by one

Returns:: Returns the new value

decrementLinesRead

public long decrementLinesRead()

Decrements the number of lines read from previous steps by one

Returns:: Returns the new value

setLinesRead

public void setLinesRead(long newLinesReadValue)

Parameters:: newLinesReadValue - the new number of lines read from previous steps

getLinesInput

public long getLinesInput()

Specified by:: getLinesInput in interface StepInterface

Returns:: Returns the number of lines read from an input source: database, file, socket, etc.

incrementLinesInput

public long incrementLinesInput()

Increments the number of lines read from an input source: database, file, socket, etc.

Returns:: the new incremented value

setLinesInput

public void setLinesInput(long newLinesInputValue)

Parameters:: newLinesInputValue - the new number of lines read from an input source: database, file, socket, etc.

getLinesOutput

public long getLinesOutput()

Specified by:: getLinesOutput in interface StepInterface

Returns:: Returns the number of lines written to an output target: database, file, socket, etc.

incrementLinesOutput

public long incrementLinesOutput()

Increments the number of lines written to an output target: database, file, socket, etc.

Returns:: the new incremented value

setLinesOutput

public void setLinesOutput(long newLinesOutputValue)

Parameters:: newLinesOutputValue - the new number of lines written to an output target: database, file, socket, etc.

getLinesWritten

public long getLinesWritten()

Specified by:: getLinesWritten in interface StepInterface

Returns:: Returns the linesWritten.

incrementLinesWritten

public long incrementLinesWritten()

Increments the number of lines written to next steps by one

Returns:: Returns the new value

decrementLinesWritten

public long decrementLinesWritten()

Decrements the number of lines written to next steps by one

Returns:: Returns the new value

setLinesWritten

public void setLinesWritten(long newLinesWrittenValue)

Parameters:: newLinesWrittenValue - the new number of lines written to next steps

getLinesUpdated

public long getLinesUpdated()

Specified by:: getLinesUpdated in interface StepInterface

Returns:: Returns the number of lines updated in an output target: database, file, socket, etc.

incrementLinesUpdated

public long incrementLinesUpdated()

Increments the number of lines updated in an output target: database, file, socket, etc.

Returns:: the new incremented value

setLinesUpdated

public void setLinesUpdated(long newLinesUpdatedValue)

Parameters:: newLinesOutputValue - the new number of lines updated in an output target: database, file, socket, etc.

getLinesRejected

public long getLinesRejected()

Specified by:: getLinesRejected in interface StepInterface

Returns:: the number of lines rejected to an error handling step

incrementLinesRejected

public long incrementLinesRejected()

Increments the number of lines rejected to an error handling step

Returns:: the new incremented value

setLinesRejected

public void setLinesRejected(long newLinesRejectedValue)

Specified by:: setLinesRejected in interface StepInterface

Parameters:: newLinesRejectedValue - lines number of lines rejected to an error handling step

getLinesSkipped

public long getLinesSkipped()

Returns:: the number of lines skipped

incrementLinesSkipped

public long incrementLinesSkipped()

Increments the number of lines skipped

Returns:: the new incremented value

setLinesSkipped

public void setLinesSkipped(long newLinesSkippedValue)

Parameters:: newLinesSkippedValue - lines number of lines skipped

getStepname

public String getStepname()

Description copied from interface: StepInterface

Get the name of the step.

Specified by:: getStepname in interface StepInterface

Returns:: the name of the step

setStepname

public void setStepname(String stepname)

getDispatcher

public Trans getDispatcher()

getStatusDescription

public String getStatusDescription()

getStepMetaInterface

public StepMetaInterface getStepMetaInterface()

Returns:: Returns the stepMetaInterface.

setStepMetaInterface

public void setStepMetaInterface(StepMetaInterface stepMetaInterface)

Parameters:: stepMetaInterface - The stepMetaInterface to set.

getStepDataInterface

public StepDataInterface getStepDataInterface()

Returns:: Returns the stepDataInterface.

setStepDataInterface

public void setStepDataInterface(StepDataInterface stepDataInterface)

Parameters:: stepDataInterface - The stepDataInterface to set.

getStepMeta

public StepMeta getStepMeta()

Specified by:: getStepMeta in interface StepInterface

Returns:: Returns the stepMeta.

setStepMeta

public void setStepMeta(StepMeta stepMeta)

Parameters:: stepMeta - The stepMeta to set.

getTransMeta

public TransMeta getTransMeta()

Returns:: Returns the transMeta.

setTransMeta

public void setTransMeta(TransMeta transMeta)

Parameters:: transMeta - The transMeta to set.

getTrans

public Trans getTrans()

Specified by:: getTrans in interface StepInterface

Returns:: Returns the trans.

putRow

public void putRow(RowMetaInterface rowMeta,
                   Object[] row)
            throws KettleStepException

putRow is used to copy a row, to the alternate rowset(s) This should get priority over everything else! (synchronized) If distribute is true, a row is copied only once to the output rowsets, otherwise copies are sent to each rowset!

Specified by:: putRow in interface StepInterface

Parameters:: row - The row to put to the destination rowset(s).; rowMeta - The row to send to the destinations steps
Throws:: KettleStepException

putRowTo

public void putRowTo(RowMetaInterface rowMeta,
                     Object[] row,
                     RowSet rowSet)
              throws KettleStepException

putRowTo is used to put a row in a certain specific RowSet.

Parameters:: rowMeta - The row meta-data to put to the destination RowSet.; row - the data to put in the RowSet; rowSet - the RoWset to put the row into.
Throws:: KettleStepException - In case something unexpected goes wrong

putError

public void putError(RowMetaInterface rowMeta,
                     Object[] row,
                     long nrErrors,
                     String errorDescriptions,
                     String fieldNames,
                     String errorCodes)
              throws KettleStepException

Throws:: KettleStepException

getRow

public Object[] getRow()
                throws KettleException

In case of getRow, we receive data from previous steps through the input rowset. In case we split the stream, we have to copy the data to the alternate splits: rowsets 1 through n.

Specified by:: getRow in interface StepInterface

Returns:: a row from the source step(s).
Throws:: KettleException

identifyErrorOutput

public void identifyErrorOutput()

Description copied from interface: StepInterface

To be used to flag an error output channel of a step prior to execution for performance reasons.

Specified by:: identifyErrorOutput in interface StepInterface

safeModeChecking

public static void safeModeChecking(RowMetaInterface referenceRowMeta,
                                    RowMetaInterface rowMeta)
                             throws KettleRowException

Throws:: KettleRowException

getRowFrom

public Object[] getRowFrom(RowSet rowSet)
                    throws KettleStepException

Throws:: KettleStepException

findInputRowSet

public RowSet findInputRowSet(String sourceStep)
                       throws KettleStepException

Throws:: KettleStepException

findInputRowSet

public RowSet findInputRowSet(String from,
                              int fromcopy,
                              String to,
                              int tocopy)

findOutputRowSet

public RowSet findOutputRowSet(String targetStep)
                        throws KettleStepException

Throws:: KettleStepException

findOutputRowSet

public RowSet findOutputRowSet(String from,
                               int fromcopy,
                               String to,
                               int tocopy)

Find an output rowset in a running transformation. It will also look at the "to" step to see if this is a mapping. If it is, it will find the appropriate rowset in that transformation.

Parameters:: from -; fromcopy -; to -; tocopy -
Returns:: The rowset or null if none is found.

setOutputDone

public void setOutputDone()

Description copied from interface: StepInterface

Signal output done to destination steps

Specified by:: setOutputDone in interface StepInterface

dispatch

public void dispatch()

This method finds the surrounding steps and rowsets for this base step. This steps keeps it's own list of rowsets (etc.) to prevent it from having to search every time.

isBasic

public boolean isBasic()

isDetailed

public boolean isDetailed()

isDebug

public boolean isDebug()

isRowLevel

public boolean isRowLevel()

logMinimal

public void logMinimal(String message)

logMinimal

public void logMinimal(String message,
                       Object... arguments)

logBasic

public void logBasic(String message)

logBasic

public void logBasic(String message,
                     Object... arguments)

logDetailed

public void logDetailed(String message)

logDetailed

public void logDetailed(String message,
                        Object... arguments)

logDebug

public void logDebug(String message)

logDebug

public void logDebug(String message,
                     Object... arguments)

logRowlevel

public void logRowlevel(String message)

logRowlevel

public void logRowlevel(String message,
                        Object... arguments)

logError

public void logError(String message)

logError

public void logError(String message,
                     Throwable e)

logError

public void logError(String message,
                     Object... arguments)

getNextClassNr

public int getNextClassNr()

outputIsDone

public boolean outputIsDone()

stopAll

public void stopAll()

Description copied from interface: StepInterface

Flags all rowsets as stopped/completed/finished.

Specified by:: stopAll in interface StepInterface

isStopped

public boolean isStopped()

Specified by:: isStopped in interface StepInterface

Returns:: True if the step is marked as stopped. Execution should stop immediate.

isRunning

public boolean isRunning()

Specified by:: isRunning in interface StepInterface

Returns:: true if the step is running after having been initialized

isPaused

public boolean isPaused()

Specified by:: isPaused in interface StepInterface

Returns:: True if the step is paused

setStopped

public void setStopped(boolean stopped)

Specified by:: setStopped in interface StepInterface

Parameters:: stopped - true if the step needs to be stopped

setRunning

public void setRunning(boolean running)

Description copied from interface: StepInterface

Flag the step as running or not

Specified by:: setRunning in interface StepInterface

Parameters:: running - the running flag to set

pauseRunning

public void pauseRunning()

Description copied from interface: StepInterface

Pause a running step

Specified by:: pauseRunning in interface StepInterface

resumeRunning

public void resumeRunning()

Description copied from interface: StepInterface

Resume a running step

Specified by:: resumeRunning in interface StepInterface

setPaused

public void setPaused(boolean paused)

setPaused

public void setPaused(AtomicBoolean paused)

isInitialising

public boolean isInitialising()

markStart

public void markStart()

Description copied from interface: StepInterface

Mark the start time of the step.

Specified by:: markStart in interface StepInterface

setInternalVariables

public void setInternalVariables()

markStop

public void markStop()

Description copied from interface: StepInterface

Mark the end time of the step.

Specified by:: markStop in interface StepInterface

getRuntime

public long getRuntime()

Specified by:: getRuntime in interface StepInterface

Returns:: The number of ms that this step has been running

buildLog

public RowMetaAndData buildLog(String sname,
                               int copynr,
                               long lines_read,
                               long lines_written,
                               long lines_updated,
                               long lines_skipped,
                               long errors,
                               Date start_date,
                               Date end_date)

getLogFields

public static final RowMetaInterface getLogFields(String comm)

toString

public String toString()

Overrides:: toString in class Object

rowsetOutputSize

public int rowsetOutputSize()

Specified by:: rowsetOutputSize in interface StepInterface

Returns:: The total amount of rows in the output buffers

rowsetInputSize

public int rowsetInputSize()

Specified by:: rowsetInputSize in interface StepInterface

Returns:: The total amount of rows in the input buffers

stopRunning

public void stopRunning(StepMetaInterface stepMetaInterface,
                        StepDataInterface stepDataInterface)
                 throws KettleException

Perform actions to stop a running step. This can be stopping running SQL queries (cancel), etc. Default it doesn't do anything.

Specified by:: stopRunning in interface StepInterface

Parameters:: stepDataInterface - The interface to the step data containing the connections, resultsets, open files, etc.; stepMetaInterface - The metadata that might be needed by the step to stop running.
Throws:: KettleException - in case something goes wrong

stopRunning

public void stopRunning()

Deprecated.

Stops running operations This method is deprecated, please use the method specifying the metadata and data interfaces.

logSummary

public void logSummary()

getStepID

public String getStepID()

Specified by:: getStepID in interface StepInterface

Returns:: the type ID of the step...

getInputRowSets

public List<RowSet> getInputRowSets()

Specified by:: getInputRowSets in interface StepInterface

Returns:: Returns the inputRowSets.

setInputRowSets

public void setInputRowSets(List<RowSet> inputRowSets)

Parameters:: inputRowSets - The inputRowSets to set.

getOutputRowSets

public List<RowSet> getOutputRowSets()

Specified by:: getOutputRowSets in interface StepInterface

Returns:: Returns the outputRowSets.

setOutputRowSets

public void setOutputRowSets(List<RowSet> outputRowSets)

Parameters:: outputRowSets - The outputRowSets to set.

isDistributed

public boolean isDistributed()

Returns:: Returns the distributed.

setDistributed

public void setDistributed(boolean distributed)

Parameters:: distributed - The distributed to set.

addRowListener

public void addRowListener(RowListener rowListener)

Description copied from interface: StepInterface

Add a rowlistener to the step allowing you to inspect (or manipulate, be careful) the rows coming in or exiting the step.

Specified by:: addRowListener in interface StepInterface

Parameters:: rowListener - the rowlistener to add

removeRowListener

public void removeRowListener(RowListener rowListener)

Description copied from interface: StepInterface

Remove a rowlistener from this step.

Specified by:: removeRowListener in interface StepInterface

Parameters:: rowListener - the rowlistener to remove

getRowListeners

public List<RowListener> getRowListeners()

Specified by:: getRowListeners in interface StepInterface

Returns:: a list of the installed RowListeners

addResultFile

public void addResultFile(ResultFile resultFile)

getResultFiles

public Map<String,ResultFile> getResultFiles()

Specified by:: getResultFiles in interface StepInterface

Returns:: The result files for this step

getStatus

public BaseStepData.StepExecutionStatus getStatus()

Specified by:: getStatus in interface StepInterface

Returns:: the description as in StepDataInterface

getPartitionID

public String getPartitionID()

Specified by:: getPartitionID in interface StepInterface

Returns:: the partitionID

setPartitionID

public void setPartitionID(String partitionID)

Specified by:: setPartitionID in interface StepInterface

Parameters:: partitionID - the partitionID to set

getPartitionTargets

public Map<String,BlockingRowSet> getPartitionTargets()

Returns:: the partitionTargets

setPartitionTargets

public void setPartitionTargets(Map<String,BlockingRowSet> partitionTargets)

Parameters:: partitionTargets - the partitionTargets to set

getRepartitioning

public int getRepartitioning()

Returns:: the repartitioning type

setRepartitioning

public void setRepartitioning(int repartitioning)

Specified by:: setRepartitioning in interface StepInterface

Parameters:: repartitioning - the repartitioning type to set

isPartitioned

public boolean isPartitioned()

Specified by:: isPartitioned in interface StepInterface

Returns:: the partitioned

setPartitioned

public void setPartitioned(boolean partitioned)

Specified by:: setPartitioned in interface StepInterface

Parameters:: partitioned - the partitioned to set

getInputRowMeta

public RowMetaInterface getInputRowMeta()

Returns:: the rowMeta

setInputRowMeta

public void setInputRowMeta(RowMetaInterface rowMeta)

Parameters:: rowMeta - the rowMeta to set

getErrorRowMeta

public RowMetaInterface getErrorRowMeta()

Returns:: the errorRowMeta

setErrorRowMeta

public void setErrorRowMeta(RowMetaInterface errorRowMeta)

Parameters:: errorRowMeta - the errorRowMeta to set

getPreviewRowMeta

public RowMetaInterface getPreviewRowMeta()

Returns:: the previewRowMeta

setPreviewRowMeta

public void setPreviewRowMeta(RowMetaInterface previewRowMeta)

Parameters:: previewRowMeta - the previewRowMeta to set

copyVariablesFrom

public void copyVariablesFrom(VariableSpace space)

Description copied from interface: VariableSpace

Copy the variables from another space, without initializing with the defaults. This does not affect any parent relationship.

Specified by:: copyVariablesFrom in interface VariableSpace

Parameters:: space - the space to copy the variables from.

environmentSubstitute

public String environmentSubstitute(String aString)

Description copied from interface: VariableSpace

Substitute the string using the current variable space.

Specified by:: environmentSubstitute in interface VariableSpace

Parameters:: aString - The string to substitute.
Returns:: The substituted string.

environmentSubstitute

public String[] environmentSubstitute(String[] aString)

Description copied from interface: VariableSpace

Replaces environment variables in an array of strings. See also: environmentSubstitute(String string)

Specified by:: environmentSubstitute in interface VariableSpace

Parameters:: aString - The array of strings that wants its variables to be replaced.
Returns:: the array with the environment variables replaced.

getParentVariableSpace

public VariableSpace getParentVariableSpace()

Description copied from interface: VariableSpace

Get the parent of the variable space.

Specified by:: getParentVariableSpace in interface VariableSpace

Returns:: the parent.

setParentVariableSpace

public void setParentVariableSpace(VariableSpace parent)

Description copied from interface: VariableSpace

Set the parent variable space

Specified by:: setParentVariableSpace in interface VariableSpace

Parameters:: parent - The parent variable space to set

getVariable

public String getVariable(String variableName,
                          String defaultValue)

Description copied from interface: VariableSpace

Get the value of a variable with a default in case the variable is not found.

Specified by:: getVariable in interface VariableSpace

Parameters:: variableName - The name of the variable; defaultValue - The default value in case the variable could not be found
Returns:: the String value of a variable

getVariable

public String getVariable(String variableName)

Description copied from interface: VariableSpace

Get the value of a variable.

Specified by:: getVariable in interface VariableSpace

Parameters:: variableName - The name of the variable
Returns:: the String value of a variable or null in case the variable could not be found.

getBooleanValueOfVariable

public boolean getBooleanValueOfVariable(String variableName,
                                         boolean defaultValue)

Description copied from interface: VariableSpace

This method returns a boolean for the new variable check boxes. If the variable name is not set or the variable name is not specified, this method simply returns the default value. If not, it convert the variable value to a boolean. "Y", "YES" and "TRUE" all convert to true. (case insensitive)

Specified by:: getBooleanValueOfVariable in interface VariableSpace

Parameters:: variableName - The variable to look up.; defaultValue - The default value to return.
Returns:
See Also:: static method ValueMeta.convertStringToBoolean()

initializeVariablesFrom

public void initializeVariablesFrom(VariableSpace parent)

Description copied from interface: VariableSpace

Initialize variable space using the defaults, copy over the variables from the parent (using copyVariablesFrom()), after this the "injected" variables should be inserted (injectVariables()). The parent is set as parent variable space.

Specified by:: initializeVariablesFrom in interface VariableSpace

Parameters:: parent - the parent to start from, or null if root.

listVariables

public String[] listVariables()

Description copied from interface: VariableSpace

List the variables (not the values) that are currently in the variable space.

Specified by:: listVariables in interface VariableSpace

Returns:: Array of String variable names.

setVariable

public void setVariable(String variableName,
                        String variableValue)

Description copied from interface: VariableSpace

Sets a variable in the Kettle Variables list.

Specified by:: setVariable in interface VariableSpace

Parameters:: variableName - The name of the variable to set; variableValue - The value of the variable to set. If the variableValue is null, the variable is cleared from the list.

shareVariablesWith

public void shareVariablesWith(VariableSpace space)

Description copied from interface: VariableSpace

Share a variable space from another variable space. This means that the object should take over the space used as argument.

Specified by:: shareVariablesWith in interface VariableSpace

Parameters:: space - Variable space to be shared.

injectVariables

public void injectVariables(Map<String,String> prop)

Description copied from interface: VariableSpace

Inject variables. The behaviour should be that the properties object will be stored and at the time the VariableSpace is initialized (or upon calling this method if the space is already initialized). After injecting the link of the properties object should be removed.

Specified by:: injectVariables in interface VariableSpace

Parameters:: prop - Properties object containing key-value pairs.

getTypeId

public String getTypeId()

Support for CheckResultSourceInterface

getSlaveNr

public int getSlaveNr()

Returns:: the unique slave number in the cluster

getClusterSize

public int getClusterSize()

Returns:: the cluster size

getUniqueStepNrAcrossSlaves

public int getUniqueStepNrAcrossSlaves()

Returns:: a unique step number across all slave servers: slaveNr * nrCopies + copyNr

getUniqueStepCountAcrossSlaves

public int getUniqueStepCountAcrossSlaves()

Returns:: the number of unique steps across all slave servers

getServerSockets

public List<ServerSocket> getServerSockets()

Returns:: the serverSockets

setServerSockets

public void setServerSockets(List<ServerSocket> serverSockets)

Parameters:: serverSockets - the serverSockets to set

setUsingThreadPriorityManagment

public void setUsingThreadPriorityManagment(boolean usingThreadPriorityManagment)

Specified by:: setUsingThreadPriorityManagment in interface StepInterface

Parameters:: usingThreadPriorityManagment - set to true to actively manage priorities of step threads

isUsingThreadPriorityManagment

public boolean isUsingThreadPriorityManagment()

Specified by:: isUsingThreadPriorityManagment in interface StepInterface

Returns:: true if we are actively managing priorities of step threads

initBeforeStart

public void initBeforeStart()
                     throws KettleStepException

This method is executed by Trans right before the threads start and right after initialization. More to the point: here we open remote output step sockets.

Specified by:: initBeforeStart in interface StepInterface

Throws:: KettleStepException - In case there is an error

getStepListeners

public List<StepListener> getStepListeners()

Returns:: the stepListeners

setStepListeners

public void setStepListeners(List<StepListener> stepListeners)

Parameters:: stepListeners - the stepListeners to set

processRow

public boolean processRow(StepMetaInterface smi,
                          StepDataInterface sdi)
                   throws KettleException

Description copied from interface: StepInterface

Perform the equivalent of processing one row. Typically this means reading a row from input (getRow()) and passing a row to output (putRow)).

Specified by:: processRow in interface StepInterface

Parameters:: smi - The steps metadata to work with; sdi - The steps temporary working data to work with (database connections, result sets, caches, temporary variables, etc.)
Returns:: false if no more rows can be processed or an error occurred.
Throws:: KettleException

canProcessOneRow

public boolean canProcessOneRow()

Description copied from interface: StepInterface

This method checks if the step is capable of processing at least one row.

For example, if a step has no input records but needs at least one to function, it will return false.

Specified by:: canProcessOneRow in interface StepInterface

Returns:: true if the step can process a row.

addStepListener

public void addStepListener(StepListener stepListener)

Description copied from interface: StepInterface

Attach a step listener to be notified when a step arrives in a certain state. (finished)

Specified by:: addStepListener in interface StepInterface

Parameters:: stepListener - The listener to add to the step

isMapping

public boolean isMapping()

Specified by:: isMapping in interface StepInterface

Returns:: true if the thread is a special mapping step

getSocketRepository

public SocketRepository getSocketRepository()

Returns:: the socketRepository

setSocketRepository

public void setSocketRepository(SocketRepository socketRepository)

Parameters:: socketRepository - the socketRepository to set

getObjectName

public String getObjectName()

Specified by:: getObjectName in interface LoggingObjectInterface

Returns:: the name

getLogChannel

public LogChannelInterface getLogChannel()

Specified by:: getLogChannel in interface HasLogChannelInterface
Specified by:: getLogChannel in interface StepInterface

Returns:: the logging channel for this step

getFilename

public String getFilename()

Specified by:: getFilename in interface LoggingObjectInterface

Returns:: the filename

getLogChannelId

public String getLogChannelId()

Specified by:: getLogChannelId in interface LoggingObjectInterface

Returns:: the log channel id

getObjectId

public ObjectId getObjectId()

Specified by:: getObjectId in interface LoggingObjectInterface

Returns:: the objectId in a repository

getObjectRevision

public ObjectRevision getObjectRevision()

Specified by:: getObjectRevision in interface LoggingObjectInterface

Returns:: the object revision in a repository

getObjectType

public LoggingObjectType getObjectType()

Specified by:: getObjectType in interface LoggingObjectInterface

Returns:: the objectType

getParent

public LoggingObjectInterface getParent()

Specified by:: getParent in interface LoggingObjectInterface

Returns:: the parent

getRepositoryDirectory

public RepositoryDirectory getRepositoryDirectory()

Specified by:: getRepositoryDirectory in interface LoggingObjectInterface

Returns:: the repositoryDirectory

getObjectCopy

public String getObjectCopy()

Specified by:: getObjectCopy in interface LoggingObjectInterface

Returns:: A string identifying a copy in a series of steps...

getLogLevel

public LogLevel getLogLevel()

Specified by:: getLogLevel in interface LoggingObjectInterface

Returns:: The logging level of the log channel of this logging object.

setLogLevel

public void setLogLevel(LogLevel logLevel)

closeQuietly

public static void closeQuietly(Closeable cl)

getContainerObjectId

public String getContainerObjectId()

Specified by:: getContainerObjectId in interface LoggingObjectInterface

Returns:: the containerObjectId

setCarteObjectId

public void setCarteObjectId(String containerObjectId)

Parameters:: containerObjectId - the containerObjectId to set

batchComplete

public void batchComplete()
                   throws KettleException

Description copied from interface: StepInterface

Calling this method will alert the step that we finished passing a batch of records to the step. Specifically for steps like "Sort Rows" it means that the buffered rows can be sorted and passed on.

Specified by:: batchComplete in interface StepInterface

Throws:: KettleException - In case an error occurs during the processing of the batch of rows.

getRemoteInputSteps

public List<RemoteStep> getRemoteInputSteps()

getRemoteOutputSteps

public List<RemoteStep> getRemoteOutputSteps()

getRegistrationDate

public Date getRegistrationDate()

Stub

Specified by:: getRegistrationDate in interface LoggingObjectInterface

Returns:: The registration date of this logging object. Null if it's not registered.

Overview

Package

Class

Use

Tree

Deprecated

Index

Help

PREV CLASS NEXT CLASS

FRAMES NO FRAMES

SUMMARY: NESTED | FIELD | CONSTR | METHOD

DETAIL: FIELD | CONSTR | METHOD

org.pentaho.di.trans.step Class BaseStep

linesRead

linesWritten

linesInput

linesOutput

linesUpdated

linesSkipped

linesRejected

first

terminator

terminator_rows

BaseStep

init

dispose

cleanup

getProcessed

setCopy

getCopy

getErrors

setErrors

getLinesRead

incrementLinesRead

decrementLinesRead

setLinesRead

getLinesInput

incrementLinesInput

setLinesInput

getLinesOutput

incrementLinesOutput

setLinesOutput

getLinesWritten

incrementLinesWritten

decrementLinesWritten

setLinesWritten

getLinesUpdated

incrementLinesUpdated

setLinesUpdated

getLinesRejected

incrementLinesRejected

setLinesRejected

getLinesSkipped

incrementLinesSkipped

setLinesSkipped

getStepname

setStepname

getDispatcher

getStatusDescription

getStepMetaInterface

setStepMetaInterface

getStepDataInterface

setStepDataInterface

getStepMeta

setStepMeta

getTransMeta

setTransMeta

getTrans

putRow

putRowTo

putError

getRow

identifyErrorOutput

safeModeChecking

getRowFrom

findInputRowSet

findInputRowSet

findOutputRowSet

findOutputRowSet

setOutputDone

dispatch

isBasic

isDetailed

isDebug

isRowLevel

logMinimal

logMinimal

logBasic

logBasic

logDetailed

logDetailed

logDebug

org.pentaho.di.trans.step
Class BaseStep