Class BaseStep
- java.lang.Object
-
- org.pentaho.di.trans.step.BaseStep
-
- All Implemented Interfaces:
org.pentaho.di.core.ExtensionDataInterface
,HasLogChannelInterface
,org.pentaho.di.core.logging.LoggingObjectInterface
,org.pentaho.di.core.logging.LoggingObjectLifecycleInterface
,org.pentaho.di.core.variables.VariableSpace
,StepInterface
- Direct Known Subclasses:
AbstractStep
,BaseDatabaseStep
,BaseFileInputStep
,BaseStreamStep
,Calculator
,Constant
,CreditCardValidator
,CsvInput
,DatabaseLookup
,DataGrid
,Denormaliser
,DetectEmptyStream
,DetectLastRow
,DummyTrans
,ExecProcess
,ExecSQL
,FieldsChangeSequence
,FieldSplitter
,FileExists
,FileLocked
,FilesFromResult
,FilesToResult
,FilterRows
,FixedInput
,Flattener
,Formula
,FuzzyMatch
,GetFileNames
,GetFilesRowsCount
,GetRepositoryNames
,GetSlaveSequence
,GetSubFolders
,GetVariable
,GroupBy
,HTTP
,HTTPPOST
,IfNull
,Injector
,Janino
,JavaFilter
,JobExecutor
,JoinRows
,LDIFInput
,LoadFileInput
,Mail
,MailInput
,MailValidator
,Mapping
,MappingInput
,MappingOutput
,MemoryGroupBy
,MergeJoin
,MergeRows
,MultiMergeJoin
,Normaliser
,NullIf
,NumberRange
,OlapInput
,ParGzipCsvInput
,PGPDecryptStream
,PGPEncryptStream
,PrioritizeStreams
,ProcessFiles
,PropertyInput
,PropertyOutput
,RandomCCNumberGenerator
,RandomValue
,RegexEval
,ReplaceString
,ReservoirSampling
,Rest
,RowGenerator
,RowsFromResult
,RowsToResult
,SampleRows
,SasInput
,Script
,ScriptValuesMod
,SecretKeyGenerator
,SelectValues
,SetValueConstant
,SetValueField
,SetVariable
,SFTPPut
,SimpleMapping
,SingleThreader
,SocketReader
,SocketWriter
,SortedMerge
,SortRows
,SplitFieldToRows
,SQLFileOutput
,SSH
,StepMetastructure
,StepsMetrics
,StreamLookup
,StringCut
,StringOperations
,SwitchCase
,SymmetricCryptoTrans
,SyslogMessage
,SystemData
,TableCompare
,TextFileInput
,TextFileOutput
,TransExecutor
,UniqueRows
,UniqueRowsByHashSet
,UnivariateStats
,UserDefinedJavaClass
,Validator
,ValueMapper
,WebService
,WebServiceAvailable
,WriteToLog
,XBaseInput
,ZipFile
public class BaseStep extends Object implements org.pentaho.di.core.variables.VariableSpace, StepInterface, org.pentaho.di.core.logging.LoggingObjectInterface, org.pentaho.di.core.ExtensionDataInterface
This class can be extended for the actual row processing of the implemented step.The implementing class can rely mostly on the base class, and has only three important methods it implements itself. The three methods implement the step lifecycle during transformation execution: initialization, row processing, and clean-up.
- Step Initialization
The init() method is called when a transformation is preparing to start execution.public boolean init(...)
Every step is given the opportunity to do one-time initialization tasks like opening files or establishing database connections. For any steps derived from BaseStep it is mandatory that super.init() is called to ensure correct behavior. The method must return true in case the step initialized correctly, it must returned false if there was an initialization error. PDI will abort the execution of a transformation in case any step returns false upon initialization.
- Row Processing
Once the transformation starts execution it enters a tight loop calling processRow() on each step until the method returns false. Each step typically reads a single row from the input stream, alters the row structure and fields and passes the row on to next steps.public boolean processRow(...)
A typical implementation queries for incoming input rows by calling getRow(), which blocks and returns a row object or null in case there is no more input. If there was an input row, the step does the necessary row processing and calls putRow() to pass the row on to the next step. If there are no more rows, the step must call setOutputDone() and return false.
Formally the method must conform to the following rules:
- If the step is done processing all rows, the method must call setOutputDone() and return false
- If the step is not done processing all rows, the method must return true. PDI will call processRow() again in this case.
- Step Clean-Up
Once the transformation is complete, PDI calls dispose() on all steps.public void dispose(...)
Steps are required to deallocate resources allocated during init() or subsequent row processing. This typically means to clear all fields of the StepDataInterface object, and to ensure that all open files or connections are properly closed. For any steps derived from BaseStep it is mandatory that super.dispose() is called to ensure correct deallocation.
-
-
Field Summary
Fields Modifier and Type Field Description protected long
deadLockCounter
Keeps track of the number of rows read for input deadlock verification.protected Map<String,Object>
extensionDataMap
boolean
first
if true then the row being processed is the first rowlong
linesInput
Deprecated.long
linesOutput
Deprecated.long
linesRead
Deprecated.long
linesRejected
Deprecated.long
linesSkipped
Deprecated.long
linesUpdated
Deprecated.long
linesWritten
Deprecated.protected org.pentaho.di.core.logging.LogChannelInterface
log
protected boolean
loggingObjectInUse
protected org.pentaho.metastore.api.IMetaStore
metaStore
The metastore that the step uses to load external elements fromprotected Repository
repository
The repository used by the step to load and reference Kettle objects with at runtimeprotected List<RowListener>
rowListeners
The list of RowListener interfacesprotected AtomicBoolean
safeStopped
boolean
terminator
List<Object[]>
terminator_rows
protected org.pentaho.di.core.variables.VariableSpace
variables
-
Constructor Summary
Constructors Constructor Description BaseStep(StepMeta stepMeta, StepDataInterface stepDataInterface, int copyNr, TransMeta transMeta, Trans trans)
This is the base step that forms that basis for all steps.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Deprecated Methods Modifier and Type Method Description void
addResultFile(org.pentaho.di.core.ResultFile resultFile)
Adds the result file.void
addRowListener(RowListener rowListener)
Add a rowlistener to the step allowing you to inspect (or manipulate, be careful) the rows coming in or exiting the step.void
addRowSetToInputRowSets(org.pentaho.di.core.RowSet rowSet)
void
addRowSetToOutputRowSets(org.pentaho.di.core.RowSet rowSet)
void
addStepListener(StepListener stepListener)
Attach a step listener to be notified when a step arrives in a certain state.void
batchComplete()
Calling this method will alert the step that we finished passing a batch of records to the step.boolean
beforeStartProcessing(StepMetaInterface smi, StepDataInterface sdi)
This method is executed by Trans right before starting processing rows.org.pentaho.di.core.RowMetaAndData
buildLog(String sname, int copynr, long lines_read, long lines_written, long lines_updated, long lines_skipped, long errors, Date start_date, Date end_date)
Builds the log.boolean
canProcessOneRow()
This method checks if the step is capable of processing at least one row.protected boolean
checkFeedback(long lines)
Check feedback.void
cleanup()
Call this method typically, after ALL the slave transformations in a clustered run have finished.protected void
clearInputRowSets()
protected void
clearOutputRowSets()
static void
closeQuietly(Closeable cl)
Close quietly.void
copyVariablesFrom(org.pentaho.di.core.variables.VariableSpace space)
long
decrementLinesRead()
Decrements the number of lines read from previous steps by onelong
decrementLinesWritten()
Decrements the number of lines written to next steps by onevoid
dispatch()
This method finds the surrounding steps and rowsets for this base step.void
dispose(StepMetaInterface smi, StepDataInterface sdi)
Dispose of this step: close files, empty logs, etc.String
environmentSubstitute(String aString)
String[]
environmentSubstitute(String[] aString)
String
environmentSubstitute(String aString, boolean escapeHexDelimiter)
String
fieldSubstitute(String aString, org.pentaho.di.core.row.RowMetaInterface rowMeta, Object[] rowData)
org.pentaho.di.core.RowSet
findInputRowSet(String sourceStep)
Find input row set.org.pentaho.di.core.RowSet
findInputRowSet(String from, int fromcopy, String to, int tocopy)
Find input row set.org.pentaho.di.core.RowSet
findOutputRowSet(String targetStep)
Find output row set.org.pentaho.di.core.RowSet
findOutputRowSet(String from, int fromcopy, String to, int tocopy)
Find an output rowset in a running transformation.boolean
getBooleanValueOfVariable(String variableName, boolean defaultValue)
int
getClusterSize()
Returns the cluster size.String
getContainerObjectId()
Returns the container object ID.int
getCopy()
int
getCurrentInputRowSetNr()
int
getCurrentOutputRowSetNr()
Trans
getDispatcher()
Gets the dispatcher.org.pentaho.di.core.row.RowMetaInterface
getErrorRowMeta()
long
getErrors()
Get the number of errorsMap<String,Object>
getExtensionDataMap()
String
getFilename()
protected org.pentaho.di.core.RowSet
getFirstInputRowSet()
org.pentaho.di.core.row.RowMetaInterface
getInputRowMeta()
List<org.pentaho.di.core.RowSet>
getInputRowSets()
long
getLinesInput()
long
getLinesOutput()
long
getLinesRead()
long
getLinesRejected()
long
getLinesSkipped()
long
getLinesUpdated()
long
getLinesWritten()
org.pentaho.di.core.logging.LogChannelInterface
getLogChannel()
String
getLogChannelId()
static org.pentaho.di.core.row.RowMetaInterface
getLogFields(String comm)
Gets the log fields.org.pentaho.di.core.logging.LogLevel
getLogLevel()
org.pentaho.metastore.api.IMetaStore
getMetaStore()
int
getNextClassNr()
Gets the next class nr.String
getObjectCopy()
org.pentaho.di.repository.ObjectId
getObjectId()
String
getObjectName()
org.pentaho.di.repository.ObjectRevision
getObjectRevision()
org.pentaho.di.core.logging.LoggingObjectType
getObjectType()
List<org.pentaho.di.core.RowSet>
getOutputRowSets()
org.pentaho.di.core.logging.LoggingObjectInterface
getParent()
org.pentaho.di.core.variables.VariableSpace
getParentVariableSpace()
String
getPartitionID()
Map<String,org.pentaho.di.core.BlockingRowSet>
getPartitionTargets()
org.pentaho.di.core.row.RowMetaInterface
getPreviewRowMeta()
long
getProcessed()
Date
getRegistrationDate()
Returns the registration dateList<RemoteStep>
getRemoteInputSteps()
Gets the remote input steps.List<RemoteStep>
getRemoteOutputSteps()
Gets the remote output steps.int
getRepartitioning()
Repository
getRepository()
org.pentaho.di.repository.RepositoryDirectory
getRepositoryDirectory()
Map<String,org.pentaho.di.core.ResultFile>
getResultFiles()
Object[]
getRow()
In case of getRow, we receive data from previous steps through the input rowset.Object[]
getRowFrom(org.pentaho.di.core.RowSet rowSet)
Gets the row from.RowHandler
getRowHandler()
List<RowListener>
getRowListeners()
long
getRuntime()
List<ServerSocket>
getServerSockets()
Returns the serverSockets.int
getSlaveNr()
Returns the unique slave number in the cluster.SocketRepository
getSocketRepository()
Retutns the socket repository.BaseStepData.StepExecutionStatus
getStatus()
String
getStatusDescription()
Gets the status description.StepDataInterface
getStepDataInterface()
String
getStepID()
List<StepListener>
getStepListeners()
Returns the step listeners.StepMeta
getStepMeta()
StepMetaInterface
getStepMetaInterface()
String
getStepname()
Get the name of the step.Trans
getTrans()
TransMeta
getTransMeta()
String
getTypeId()
Returns the step ID via the getStepID() method call.int
getUniqueStepCountAcrossSlaves()
Returns the number of unique steps across all slave servers.int
getUniqueStepNrAcrossSlaves()
Returns a unique step number across all slave servers: slaveNr * nrCopies + copyNr.String
getVariable(String variableName)
String
getVariable(String variableName, String defaultValue)
Object[]
handleGetRowFrom(org.pentaho.di.core.RowSet rowSet)
void
handlePutRowTo(org.pentaho.di.core.row.RowMetaInterface rowMeta, Object[] row, org.pentaho.di.core.RowSet rowSet)
void
identifyErrorOutput()
To be used to flag an error output channel of a step prior to execution for performance reasons.long
incrementLinesInput()
Increments the number of lines read from an input source: database, file, socket, etc.long
incrementLinesOutput()
Increments the number of lines written to an output target: database, file, socket, etc.long
incrementLinesRead()
Increments the number of lines read from previous steps by onelong
incrementLinesRejected()
Increments the number of lines rejected to an error handling steplong
incrementLinesSkipped()
Increments the number of lines skippedlong
incrementLinesUpdated()
Increments the number of lines updated in an output target: database, file, socket, etc.long
incrementLinesWritten()
Increments the number of lines written to next steps by oneboolean
init(StepMetaInterface smi, StepDataInterface sdi)
Initialize and do work where other steps need to wait for...void
initBeforeStart()
This method is executed by Trans right before the threads start and right after initialization.void
initializeVariablesFrom(org.pentaho.di.core.variables.VariableSpace parent)
void
injectVariables(Map<String,String> prop)
boolean
isBasic()
Checks if is basic.boolean
isDebug()
Checks if is debug.boolean
isDetailed()
Checks if is detailed.boolean
isDistributed()
boolean
isForcingSeparateLogging()
boolean
isGatheringMetrics()
boolean
isInitialising()
Checks if is initialising.boolean
isLoggingObjectInUse()
boolean
isMapping()
boolean
isPartitioned()
boolean
isPaused()
boolean
isRowLevel()
Checks if is row level.boolean
isRunning()
boolean
isSafeStopped()
boolean
isStopped()
boolean
isUsingThreadPriorityManagment()
Retusn true if we are actively managing priorities of step threads.String[]
listVariables()
void
logBasic(String message)
Log basic.void
logBasic(String message, Object... arguments)
Log basic.void
logDebug(String message)
Log debug.void
logDebug(String message, Object... arguments)
Log debug.void
logDetailed(String message)
Log detailed.void
logDetailed(String message, Object... arguments)
Log detailed.void
logError(String message)
Log error.void
logError(String message, Object... arguments)
Log error.void
logError(String message, Throwable e)
Log error.void
logMinimal(String message)
Log minimal.void
logMinimal(String message, Object... arguments)
Log minimal.void
logRowlevel(String message)
Log rowlevel.void
logRowlevel(String message, Object... arguments)
Log rowlevel.void
logSummary()
Log summary.void
markStart()
Mark the start time of the step.void
markStop()
Mark the end time of the step.protected void
openRemoteInputStepSocketsOnce()
Opens socket connections to the remote input steps of this step.protected void
openRemoteOutputStepSocketsOnce()
Opens socket connections to the remote output steps of this step.boolean
outputIsDone()
Output is done.void
pauseRunning()
Pause a running stepboolean
processRow(StepMetaInterface smi, StepDataInterface sdi)
Perform the equivalent of processing one row.void
putError(org.pentaho.di.core.row.RowMetaInterface rowMeta, Object[] row, long nrErrors, String errorDescriptions, String fieldNames, String errorCodes)
Put error.void
putRow(org.pentaho.di.core.row.RowMetaInterface rowMeta, Object[] row)
putRow is used to copy a row, to the alternate rowset(s) This should get priority over everything else! (synchronized) If distribute is true, a row is copied only once to the output rowsets, otherwise copies are sent to each rowset!void
putRowTo(org.pentaho.di.core.row.RowMetaInterface rowMeta, Object[] row, org.pentaho.di.core.RowSet rowSet)
putRowTo is used to put a row in a certain specific RowSet.void
removeRowListener(RowListener rowListener)
Remove a rowlistener from this step.void
resumeRunning()
Resume a running stepint
rowsetInputSize()
int
rowsetOutputSize()
protected void
safeModeChecking(org.pentaho.di.core.row.RowMetaInterface row)
Safe mode checking.static void
safeModeChecking(org.pentaho.di.core.row.RowMetaInterface referenceRowMeta, org.pentaho.di.core.row.RowMetaInterface rowMeta)
Safe mode checking.void
setCarteObjectId(String containerObjectId)
Sets the container object ID.void
setCopy(int cop)
Sets the copy.void
setCurrentInputRowSetNr(int index)
void
setCurrentOutputRowSetNr(int index)
void
setDistributed(boolean distributed)
void
setErrorRowMeta(org.pentaho.di.core.row.RowMetaInterface errorRowMeta)
void
setErrors(long e)
Sets the number of errorsvoid
setForcingSeparateLogging(boolean forcingSeparateLogging)
void
setGatheringMetrics(boolean gatheringMetrics)
void
setInputRowMeta(org.pentaho.di.core.row.RowMetaInterface rowMeta)
void
setInputRowSets(List<org.pentaho.di.core.RowSet> inputRowSets)
void
setInternalVariables()
Sets the internal variables.void
setLinesInput(long newLinesInputValue)
void
setLinesOutput(long newLinesOutputValue)
void
setLinesRead(long newLinesReadValue)
void
setLinesRejected(long newLinesRejectedValue)
void
setLinesSkipped(long newLinesSkippedValue)
void
setLinesUpdated(long newLinesUpdatedValue)
void
setLinesWritten(long newLinesWrittenValue)
void
setLoggingObjectInUse(boolean inUse)
void
setLogLevel(org.pentaho.di.core.logging.LogLevel logLevel)
Sets the log level.void
setMetaStore(org.pentaho.metastore.api.IMetaStore metaStore)
Pass along the metastore to use when loading external elements at runtime.void
setOutputDone()
Signal output done to destination stepsvoid
setOutputRowSets(List<org.pentaho.di.core.RowSet> outputRowSets)
void
setParentVariableSpace(org.pentaho.di.core.variables.VariableSpace parent)
void
setPartitioned(boolean partitioned)
void
setPartitionID(String partitionID)
void
setPartitionTargets(Map<String,org.pentaho.di.core.BlockingRowSet> partitionTargets)
void
setPaused(boolean paused)
Sets the paused.void
setPaused(AtomicBoolean paused)
Sets the paused.void
setPreviewRowMeta(org.pentaho.di.core.row.RowMetaInterface previewRowMeta)
void
setRepartitioning(int repartitioning)
void
setRepository(Repository repository)
void
setRowHandler(RowHandler rowHandler)
RowHandler controls how getRow/putRow are handled.void
setRunning(boolean running)
Flag the step as running or notvoid
setSafeStopped(boolean stopped)
void
setServerSockets(List<ServerSocket> serverSockets)
void
setSocketRepository(SocketRepository socketRepository)
Sets the socket repository.void
setStepDataInterface(StepDataInterface stepDataInterface)
void
setStepListeners(List<StepListener> stepListeners)
Sets the step listeners.void
setStepMeta(StepMeta stepMeta)
void
setStepMetaInterface(StepMetaInterface stepMetaInterface)
void
setStepname(String stepname)
Sets the stepname.void
setStopped(boolean stopped)
void
setTransMeta(TransMeta transMeta)
void
setUsingThreadPriorityManagment(boolean usingThreadPriorityManagment)
Set to true to actively manage priorities of step threads.void
setVariable(String variableName, String variableValue)
void
shareVariablesWith(org.pentaho.di.core.variables.VariableSpace space)
void
stopAll()
Flags all rowsets as stopped/completed/finished.void
stopRunning()
Deprecated.void
stopRunning(StepMetaInterface stepMetaInterface, StepDataInterface stepDataInterface)
Perform actions to stop a running step.protected void
swapFirstInputRowSetIfExists(String stepName)
String
toString()
protected void
verifyInputDeadLock()
- A step sees that it can't get a new row from input in the step.protected void
waitUntilTransformationIsStarted()
Wait until the transformation is completely running and all threads have been started.-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
-
Methods inherited from interface org.pentaho.di.core.logging.LoggingObjectLifecycleInterface
callAfterLog, callBeforeLog
-
Methods inherited from interface org.pentaho.di.trans.step.StepInterface
afterFinishProcessing, subStatuses
-
-
-
-
Field Detail
-
variables
protected org.pentaho.di.core.variables.VariableSpace variables
-
log
protected org.pentaho.di.core.logging.LogChannelInterface log
-
loggingObjectInUse
protected boolean loggingObjectInUse
-
linesRead
@Deprecated public long linesRead
Deprecated.nr of lines read from previous step(s)
-
linesWritten
@Deprecated public long linesWritten
Deprecated.nr of lines written to next step(s)
-
linesInput
@Deprecated public long linesInput
Deprecated.nr of lines read from file or database
-
linesOutput
@Deprecated public long linesOutput
Deprecated.nr of lines written to file or database
-
linesUpdated
@Deprecated public long linesUpdated
Deprecated.nr of updates in a database table or file
-
linesSkipped
@Deprecated public long linesSkipped
Deprecated.nr of lines skipped
-
linesRejected
@Deprecated public long linesRejected
Deprecated.total sleep time in ns caused by an empty input buffer (previous step is slow)
-
safeStopped
protected AtomicBoolean safeStopped
-
first
public boolean first
if true then the row being processed is the first row
-
terminator
public boolean terminator
-
rowListeners
protected List<RowListener> rowListeners
The list of RowListener interfaces
-
deadLockCounter
protected long deadLockCounter
Keeps track of the number of rows read for input deadlock verification.
-
repository
protected Repository repository
The repository used by the step to load and reference Kettle objects with at runtime
-
metaStore
protected org.pentaho.metastore.api.IMetaStore metaStore
The metastore that the step uses to load external elements from
-
-
Constructor Detail
-
BaseStep
public BaseStep(StepMeta stepMeta, StepDataInterface stepDataInterface, int copyNr, TransMeta transMeta, Trans trans)
This is the base step that forms that basis for all steps. You can derive from this class to implement your own steps.- Parameters:
stepMeta
- The StepMeta object to run.stepDataInterface
- the data object to store temporary data, database connections, caches, result sets, hashtables etc.copyNr
- The copynumber for this step.transMeta
- The TransInfo of which the step stepMeta is part of.trans
- The (running) transformation to obtain information shared among the steps.
-
-
Method Detail
-
init
public boolean init(StepMetaInterface smi, StepDataInterface sdi)
Description copied from interface:StepInterface
Initialize and do work where other steps need to wait for...- Specified by:
init
in interfaceStepInterface
- Parameters:
smi
- The metadata to work withsdi
- The data to initialize
-
dispose
public void dispose(StepMetaInterface smi, StepDataInterface sdi)
Description copied from interface:StepInterface
Dispose of this step: close files, empty logs, etc.- Specified by:
dispose
in interfaceStepInterface
- Parameters:
smi
- The metadata to work withsdi
- The data to dispose of
-
cleanup
public void cleanup()
Description copied from interface:StepInterface
Call this method typically, after ALL the slave transformations in a clustered run have finished.- Specified by:
cleanup
in interfaceStepInterface
-
getProcessed
public long getProcessed()
- Specified by:
getProcessed
in interfaceStepInterface
- Returns:
- The number of "processed" lines of a step. Well, a representable metric for that anyway.
-
setCopy
public void setCopy(int cop)
Sets the copy.- Parameters:
cop
- the new copy
-
getCopy
public int getCopy()
- Specified by:
getCopy
in interfaceStepInterface
- Returns:
- The steps copy number (default 0)
-
getErrors
public long getErrors()
Description copied from interface:StepInterface
Get the number of errors- Specified by:
getErrors
in interfaceStepInterface
- Returns:
- the number of errors
-
setErrors
public void setErrors(long e)
Description copied from interface:StepInterface
Sets the number of errors- Specified by:
setErrors
in interfaceStepInterface
- Parameters:
e
- the number of errors to set
-
getLinesRead
public long getLinesRead()
- Specified by:
getLinesRead
in interfaceStepInterface
- Returns:
- Returns the number of lines read from previous steps
-
incrementLinesRead
public long incrementLinesRead()
Increments the number of lines read from previous steps by one- Returns:
- Returns the new value
-
decrementLinesRead
public long decrementLinesRead()
Decrements the number of lines read from previous steps by one- Returns:
- Returns the new value
-
setLinesRead
public void setLinesRead(long newLinesReadValue)
- Parameters:
newLinesReadValue
- the new number of lines read from previous steps
-
getLinesInput
public long getLinesInput()
- Specified by:
getLinesInput
in interfaceStepInterface
- Returns:
- Returns the number of lines read from an input source: database, file, socket, etc.
-
incrementLinesInput
public long incrementLinesInput()
Increments the number of lines read from an input source: database, file, socket, etc.- Returns:
- the new incremented value
-
setLinesInput
public void setLinesInput(long newLinesInputValue)
- Parameters:
newLinesInputValue
- the new number of lines read from an input source: database, file, socket, etc.
-
getLinesOutput
public long getLinesOutput()
- Specified by:
getLinesOutput
in interfaceStepInterface
- Returns:
- Returns the number of lines written to an output target: database, file, socket, etc.
-
incrementLinesOutput
public long incrementLinesOutput()
Increments the number of lines written to an output target: database, file, socket, etc.- Returns:
- the new incremented value
-
setLinesOutput
public void setLinesOutput(long newLinesOutputValue)
- Parameters:
newLinesOutputValue
- the new number of lines written to an output target: database, file, socket, etc.
-
getLinesWritten
public long getLinesWritten()
- Specified by:
getLinesWritten
in interfaceStepInterface
- Returns:
- Returns the linesWritten.
-
incrementLinesWritten
public long incrementLinesWritten()
Increments the number of lines written to next steps by one- Returns:
- Returns the new value
-
decrementLinesWritten
public long decrementLinesWritten()
Decrements the number of lines written to next steps by one- Returns:
- Returns the new value
-
setLinesWritten
public void setLinesWritten(long newLinesWrittenValue)
- Parameters:
newLinesWrittenValue
- the new number of lines written to next steps
-
getLinesUpdated
public long getLinesUpdated()
- Specified by:
getLinesUpdated
in interfaceStepInterface
- Returns:
- Returns the number of lines updated in an output target: database, file, socket, etc.
-
incrementLinesUpdated
public long incrementLinesUpdated()
Increments the number of lines updated in an output target: database, file, socket, etc.- Returns:
- the new incremented value
-
setLinesUpdated
public void setLinesUpdated(long newLinesUpdatedValue)
- Parameters:
newLinesUpdatedValue
- the new number of lines updated in an output target: database, file, socket, etc.
-
getLinesRejected
public long getLinesRejected()
- Specified by:
getLinesRejected
in interfaceStepInterface
- Returns:
- the number of lines rejected to an error handling step
-
incrementLinesRejected
public long incrementLinesRejected()
Increments the number of lines rejected to an error handling step- Returns:
- the new incremented value
-
setLinesRejected
public void setLinesRejected(long newLinesRejectedValue)
- Specified by:
setLinesRejected
in interfaceStepInterface
- Parameters:
newLinesRejectedValue
- lines number of lines rejected to an error handling step
-
getLinesSkipped
public long getLinesSkipped()
- Returns:
- the number of lines skipped
-
incrementLinesSkipped
public long incrementLinesSkipped()
Increments the number of lines skipped- Returns:
- the new incremented value
-
setLinesSkipped
public void setLinesSkipped(long newLinesSkippedValue)
- Parameters:
newLinesSkippedValue
- lines number of lines skipped
-
getStepname
public String getStepname()
Description copied from interface:StepInterface
Get the name of the step.- Specified by:
getStepname
in interfaceStepInterface
- Returns:
- the name of the step
-
setStepname
public void setStepname(String stepname)
Sets the stepname.- Parameters:
stepname
- the new stepname
-
getDispatcher
public Trans getDispatcher()
Gets the dispatcher.- Returns:
- the dispatcher
-
getStatusDescription
public String getStatusDescription()
Gets the status description.- Returns:
- the status description
-
getStepMetaInterface
public StepMetaInterface getStepMetaInterface()
- Returns:
- Returns the stepMetaInterface.
-
setStepMetaInterface
public void setStepMetaInterface(StepMetaInterface stepMetaInterface)
- Parameters:
stepMetaInterface
- The stepMetaInterface to set.
-
getStepDataInterface
public StepDataInterface getStepDataInterface()
- Returns:
- Returns the stepDataInterface.
-
setStepDataInterface
public void setStepDataInterface(StepDataInterface stepDataInterface)
- Parameters:
stepDataInterface
- The stepDataInterface to set.
-
getStepMeta
public StepMeta getStepMeta()
- Specified by:
getStepMeta
in interfaceStepInterface
- Returns:
- Returns the stepMeta.
-
setStepMeta
public void setStepMeta(StepMeta stepMeta)
- Parameters:
stepMeta
- The stepMeta to set.
-
getTransMeta
public TransMeta getTransMeta()
- Returns:
- Returns the transMeta.
-
setTransMeta
public void setTransMeta(TransMeta transMeta)
- Parameters:
transMeta
- The transMeta to set.
-
getTrans
public Trans getTrans()
- Specified by:
getTrans
in interfaceStepInterface
- Returns:
- Returns the trans.
-
putRow
public void putRow(org.pentaho.di.core.row.RowMetaInterface rowMeta, Object[] row) throws org.pentaho.di.core.exception.KettleStepException
putRow is used to copy a row, to the alternate rowset(s) This should get priority over everything else! (synchronized) If distribute is true, a row is copied only once to the output rowsets, otherwise copies are sent to each rowset!- Specified by:
putRow
in interfaceStepInterface
- Parameters:
row
- The row to put to the destination rowset(s).rowMeta
- The row to send to the destinations steps- Throws:
org.pentaho.di.core.exception.KettleStepException
-
putRowTo
public void putRowTo(org.pentaho.di.core.row.RowMetaInterface rowMeta, Object[] row, org.pentaho.di.core.RowSet rowSet) throws org.pentaho.di.core.exception.KettleStepException
putRowTo is used to put a row in a certain specific RowSet.- Parameters:
rowMeta
- The row meta-data to put to the destination RowSet.row
- the data to put in the RowSetrowSet
- the RoWset to put the row into.- Throws:
org.pentaho.di.core.exception.KettleStepException
- In case something unexpected goes wrong
-
handlePutRowTo
public void handlePutRowTo(org.pentaho.di.core.row.RowMetaInterface rowMeta, Object[] row, org.pentaho.di.core.RowSet rowSet) throws org.pentaho.di.core.exception.KettleStepException
- Throws:
org.pentaho.di.core.exception.KettleStepException
-
putError
public void putError(org.pentaho.di.core.row.RowMetaInterface rowMeta, Object[] row, long nrErrors, String errorDescriptions, String fieldNames, String errorCodes) throws org.pentaho.di.core.exception.KettleStepException
Put error.- Parameters:
rowMeta
- the row metarow
- the rownrErrors
- the nr errorserrorDescriptions
- the error descriptionsfieldNames
- the field nameserrorCodes
- the error codes- Throws:
org.pentaho.di.core.exception.KettleStepException
- the kettle step exception
-
waitUntilTransformationIsStarted
protected void waitUntilTransformationIsStarted()
Wait until the transformation is completely running and all threads have been started.
-
getRow
public Object[] getRow() throws org.pentaho.di.core.exception.KettleException
In case of getRow, we receive data from previous steps through the input rowset. In case we split the stream, we have to copy the data to the alternate splits: rowsets 1 through n.- Specified by:
getRow
in interfaceStepInterface
- Returns:
- a row from the source step(s).
- Throws:
org.pentaho.di.core.exception.KettleException
-
setRowHandler
public void setRowHandler(RowHandler rowHandler)
RowHandler controls how getRow/putRow are handled. The default RowHandler will simply callhandleGetRow()
andhandlePutRow(RowMetaInterface, Object[])
-
getRowHandler
public RowHandler getRowHandler()
-
openRemoteInputStepSocketsOnce
protected void openRemoteInputStepSocketsOnce() throws org.pentaho.di.core.exception.KettleStepException
Opens socket connections to the remote input steps of this step.
This method should be used by steps that don't call getRow() first in which it is executed automatically.
This method should be called before any data is read from previous steps.
This action is executed only once.- Throws:
org.pentaho.di.core.exception.KettleStepException
-
openRemoteOutputStepSocketsOnce
protected void openRemoteOutputStepSocketsOnce() throws org.pentaho.di.core.exception.KettleStepException
Opens socket connections to the remote output steps of this step.
This method is called in method initBeforeStart() because it needs to connect to the server sockets (remote steps) as soon as possible to avoid time-out situations.
This action is executed only once.- Throws:
org.pentaho.di.core.exception.KettleStepException
-
safeModeChecking
protected void safeModeChecking(org.pentaho.di.core.row.RowMetaInterface row) throws org.pentaho.di.core.exception.KettleRowException
Safe mode checking.- Parameters:
row
- the row- Throws:
org.pentaho.di.core.exception.KettleRowException
- the kettle row exception
-
identifyErrorOutput
public void identifyErrorOutput()
Description copied from interface:StepInterface
To be used to flag an error output channel of a step prior to execution for performance reasons.- Specified by:
identifyErrorOutput
in interfaceStepInterface
-
safeModeChecking
public static void safeModeChecking(org.pentaho.di.core.row.RowMetaInterface referenceRowMeta, org.pentaho.di.core.row.RowMetaInterface rowMeta) throws org.pentaho.di.core.exception.KettleRowException
Safe mode checking.- Parameters:
referenceRowMeta
- the reference row metarowMeta
- the row meta- Throws:
org.pentaho.di.core.exception.KettleRowException
- the kettle row exception
-
getRowFrom
public Object[] getRowFrom(org.pentaho.di.core.RowSet rowSet) throws org.pentaho.di.core.exception.KettleStepException
Gets the row from.- Parameters:
rowSet
- the row set- Returns:
- the row from
- Throws:
org.pentaho.di.core.exception.KettleStepException
- the kettle step exception
-
handleGetRowFrom
public Object[] handleGetRowFrom(org.pentaho.di.core.RowSet rowSet) throws org.pentaho.di.core.exception.KettleStepException
- Throws:
org.pentaho.di.core.exception.KettleStepException
-
verifyInputDeadLock
protected void verifyInputDeadLock() throws org.pentaho.di.core.exception.KettleStepException
- A step sees that it can't get a new row from input in the step. - Then it verifies that there is more than one input row set and that at least one is full and at least one is empty. - Then it finds a step in the transformation (situated before the reader step) which has at least one full and one empty output row set. - If this situation presents itself and if it happens twice with the same rows read count (meaning: stalled reading step) we throw an exception. For the attached example that exception is:- Throws:
org.pentaho.di.core.exception.KettleStepException
-
findInputRowSet
public org.pentaho.di.core.RowSet findInputRowSet(String sourceStep) throws org.pentaho.di.core.exception.KettleStepException
Find input row set.- Parameters:
sourceStep
- the source step- Returns:
- the row set
- Throws:
org.pentaho.di.core.exception.KettleStepException
- the kettle step exception
-
findInputRowSet
public org.pentaho.di.core.RowSet findInputRowSet(String from, int fromcopy, String to, int tocopy)
Find input row set.- Parameters:
from
- the fromfromcopy
- the fromcopyto
- the totocopy
- the tocopy- Returns:
- the row set
-
findOutputRowSet
public org.pentaho.di.core.RowSet findOutputRowSet(String targetStep) throws org.pentaho.di.core.exception.KettleStepException
Find output row set.- Parameters:
targetStep
- the target step- Returns:
- the row set
- Throws:
org.pentaho.di.core.exception.KettleStepException
- the kettle step exception
-
findOutputRowSet
public org.pentaho.di.core.RowSet findOutputRowSet(String from, int fromcopy, String to, int tocopy)
Find an output rowset in a running transformation. It will also look at the "to" step to see if this is a mapping. If it is, it will find the appropriate rowset in that transformation.- Parameters:
from
-fromcopy
-to
-tocopy
-- Returns:
- The rowset or null if none is found.
-
setOutputDone
public void setOutputDone()
Description copied from interface:StepInterface
Signal output done to destination steps- Specified by:
setOutputDone
in interfaceStepInterface
-
dispatch
public void dispatch()
This method finds the surrounding steps and rowsets for this base step. This steps keeps it's own list of rowsets (etc.) to prevent it from having to search every time.Note that all rowsets input and output is already created by transformation itself. So in this place we will look and choose which rowsets will be used by this particular step.
We will collect all input rowsets and output rowsets so step will be able to read input data, and write to the output.
Steps can run in multiple copies, on in partitioned fashion. For this case we should take in account that in different cases we should take in account one to one, one to many and other cases properly.
-
isBasic
public boolean isBasic()
Checks if is basic.- Returns:
- true, if is basic
-
isDetailed
public boolean isDetailed()
Checks if is detailed.- Returns:
- true, if is detailed
-
isDebug
public boolean isDebug()
Checks if is debug.- Returns:
- true, if is debug
-
isRowLevel
public boolean isRowLevel()
Checks if is row level.- Returns:
- true, if is row level
-
logMinimal
public void logMinimal(String message)
Log minimal.- Parameters:
message
- the message
-
logMinimal
public void logMinimal(String message, Object... arguments)
Log minimal.- Parameters:
message
- the messagearguments
- the arguments
-
logBasic
public void logBasic(String message)
Log basic.- Parameters:
message
- the message
-
logBasic
public void logBasic(String message, Object... arguments)
Log basic.- Parameters:
message
- the messagearguments
- the arguments
-
logDetailed
public void logDetailed(String message)
Log detailed.- Parameters:
message
- the message
-
logDetailed
public void logDetailed(String message, Object... arguments)
Log detailed.- Parameters:
message
- the messagearguments
- the arguments
-
logDebug
public void logDebug(String message)
Log debug.- Parameters:
message
- the message
-
logDebug
public void logDebug(String message, Object... arguments)
Log debug.- Parameters:
message
- the messagearguments
- the arguments
-
logRowlevel
public void logRowlevel(String message)
Log rowlevel.- Parameters:
message
- the message
-
logRowlevel
public void logRowlevel(String message, Object... arguments)
Log rowlevel.- Parameters:
message
- the messagearguments
- the arguments
-
logError
public void logError(String message)
Log error.- Parameters:
message
- the message
-
logError
public void logError(String message, Throwable e)
Log error.- Parameters:
message
- the messagee
- the e
-
logError
public void logError(String message, Object... arguments)
Log error.- Parameters:
message
- the messagearguments
- the arguments
-
getNextClassNr
public int getNextClassNr()
Gets the next class nr.- Returns:
- the next class nr
-
outputIsDone
public boolean outputIsDone()
Output is done.- Returns:
- true, if successful
-
stopAll
public void stopAll()
Description copied from interface:StepInterface
Flags all rowsets as stopped/completed/finished.- Specified by:
stopAll
in interfaceStepInterface
-
isStopped
public boolean isStopped()
- Specified by:
isStopped
in interfaceStepInterface
- Returns:
- True if the step is marked as stopped. Execution should stop immediate.
-
isRunning
public boolean isRunning()
- Specified by:
isRunning
in interfaceStepInterface
- Returns:
- true if the step is running after having been initialized
-
isPaused
public boolean isPaused()
- Specified by:
isPaused
in interfaceStepInterface
- Returns:
- True if the step is paused
-
setStopped
public void setStopped(boolean stopped)
- Specified by:
setStopped
in interfaceStepInterface
- Parameters:
stopped
- true if the step needs to be stopped
-
setSafeStopped
public void setSafeStopped(boolean stopped)
- Specified by:
setSafeStopped
in interfaceStepInterface
- Parameters:
stopped
- true if the step needs to be safe stopped
-
isSafeStopped
public boolean isSafeStopped()
- Specified by:
isSafeStopped
in interfaceStepInterface
- Returns:
- true if step is safe stopped.
-
setRunning
public void setRunning(boolean running)
Description copied from interface:StepInterface
Flag the step as running or not- Specified by:
setRunning
in interfaceStepInterface
- Parameters:
running
- the running flag to set
-
pauseRunning
public void pauseRunning()
Description copied from interface:StepInterface
Pause a running step- Specified by:
pauseRunning
in interfaceStepInterface
-
resumeRunning
public void resumeRunning()
Description copied from interface:StepInterface
Resume a running step- Specified by:
resumeRunning
in interfaceStepInterface
-
setPaused
public void setPaused(boolean paused)
Sets the paused.- Parameters:
paused
- the new paused
-
setPaused
public void setPaused(AtomicBoolean paused)
Sets the paused.- Parameters:
paused
- the new paused
-
isInitialising
public boolean isInitialising()
Checks if is initialising.- Returns:
- true, if is initialising
-
markStart
public void markStart()
Description copied from interface:StepInterface
Mark the start time of the step.- Specified by:
markStart
in interfaceStepInterface
-
setInternalVariables
public void setInternalVariables()
Sets the internal variables.
-
markStop
public void markStop()
Description copied from interface:StepInterface
Mark the end time of the step.- Specified by:
markStop
in interfaceStepInterface
-
getRuntime
public long getRuntime()
- Specified by:
getRuntime
in interfaceStepInterface
- Returns:
- The number of ms that this step has been running
-
buildLog
public org.pentaho.di.core.RowMetaAndData buildLog(String sname, int copynr, long lines_read, long lines_written, long lines_updated, long lines_skipped, long errors, Date start_date, Date end_date)
Builds the log.- Parameters:
sname
- the snamecopynr
- the copynrlines_read
- the lines_readlines_written
- the lines_writtenlines_updated
- the lines_updatedlines_skipped
- the lines_skippederrors
- the errorsstart_date
- the start_dateend_date
- the end_date- Returns:
- the row meta and data
-
getLogFields
public static final org.pentaho.di.core.row.RowMetaInterface getLogFields(String comm)
Gets the log fields.- Parameters:
comm
- the comm- Returns:
- the log fields
-
rowsetOutputSize
public int rowsetOutputSize()
- Specified by:
rowsetOutputSize
in interfaceStepInterface
- Returns:
- The total amount of rows in the output buffers
-
rowsetInputSize
public int rowsetInputSize()
- Specified by:
rowsetInputSize
in interfaceStepInterface
- Returns:
- The total amount of rows in the input buffers
-
stopRunning
public void stopRunning(StepMetaInterface stepMetaInterface, StepDataInterface stepDataInterface) throws org.pentaho.di.core.exception.KettleException
Perform actions to stop a running step. This can be stopping running SQL queries (cancel), etc. Default it doesn't do anything.- Specified by:
stopRunning
in interfaceStepInterface
- Parameters:
stepDataInterface
- The interface to the step data containing the connections, resultsets, open files, etc.stepMetaInterface
- The metadata that might be needed by the step to stop running.- Throws:
org.pentaho.di.core.exception.KettleException
- in case something goes wrong
-
stopRunning
@Deprecated public void stopRunning()
Deprecated.Stops running operations This method is deprecated, please use the method specifying the metadata and data interfaces.
-
logSummary
public void logSummary()
Log summary.
-
getStepID
public String getStepID()
- Specified by:
getStepID
in interfaceStepInterface
- Returns:
- the type ID of the step...
-
getInputRowSets
public List<org.pentaho.di.core.RowSet> getInputRowSets()
- Specified by:
getInputRowSets
in interfaceStepInterface
- Returns:
- Returns the inputRowSets.
-
addRowSetToInputRowSets
public void addRowSetToInputRowSets(org.pentaho.di.core.RowSet rowSet)
- Specified by:
addRowSetToInputRowSets
in interfaceStepInterface
-
getFirstInputRowSet
protected org.pentaho.di.core.RowSet getFirstInputRowSet()
-
clearInputRowSets
protected void clearInputRowSets()
-
swapFirstInputRowSetIfExists
protected void swapFirstInputRowSetIfExists(String stepName)
-
setInputRowSets
public void setInputRowSets(List<org.pentaho.di.core.RowSet> inputRowSets)
- Parameters:
inputRowSets
- The inputRowSets to set.
-
getOutputRowSets
public List<org.pentaho.di.core.RowSet> getOutputRowSets()
- Specified by:
getOutputRowSets
in interfaceStepInterface
- Returns:
- Returns the outputRowSets.
-
addRowSetToOutputRowSets
public void addRowSetToOutputRowSets(org.pentaho.di.core.RowSet rowSet)
- Specified by:
addRowSetToOutputRowSets
in interfaceStepInterface
-
clearOutputRowSets
protected void clearOutputRowSets()
-
setOutputRowSets
public void setOutputRowSets(List<org.pentaho.di.core.RowSet> outputRowSets)
- Parameters:
outputRowSets
- The outputRowSets to set.
-
isDistributed
public boolean isDistributed()
- Returns:
- Returns the distributed.
-
setDistributed
public void setDistributed(boolean distributed)
- Parameters:
distributed
- The distributed to set.
-
addRowListener
public void addRowListener(RowListener rowListener)
Description copied from interface:StepInterface
Add a rowlistener to the step allowing you to inspect (or manipulate, be careful) the rows coming in or exiting the step.- Specified by:
addRowListener
in interfaceStepInterface
- Parameters:
rowListener
- the rowlistener to add
-
removeRowListener
public void removeRowListener(RowListener rowListener)
Description copied from interface:StepInterface
Remove a rowlistener from this step.- Specified by:
removeRowListener
in interfaceStepInterface
- Parameters:
rowListener
- the rowlistener to remove
-
getRowListeners
public List<RowListener> getRowListeners()
- Specified by:
getRowListeners
in interfaceStepInterface
- Returns:
- a list of the installed RowListeners
-
addResultFile
public void addResultFile(org.pentaho.di.core.ResultFile resultFile)
Adds the result file.- Parameters:
resultFile
- the result file
-
getResultFiles
public Map<String,org.pentaho.di.core.ResultFile> getResultFiles()
- Specified by:
getResultFiles
in interfaceStepInterface
- Returns:
- The result files for this step
-
getStatus
public BaseStepData.StepExecutionStatus getStatus()
- Specified by:
getStatus
in interfaceStepInterface
- Returns:
- the description as in
StepDataInterface
-
getPartitionID
public String getPartitionID()
- Specified by:
getPartitionID
in interfaceStepInterface
- Returns:
- the partitionID
-
setPartitionID
public void setPartitionID(String partitionID)
- Specified by:
setPartitionID
in interfaceStepInterface
- Parameters:
partitionID
- the partitionID to set
-
getPartitionTargets
public Map<String,org.pentaho.di.core.BlockingRowSet> getPartitionTargets()
- Returns:
- the partitionTargets
-
setPartitionTargets
public void setPartitionTargets(Map<String,org.pentaho.di.core.BlockingRowSet> partitionTargets)
- Parameters:
partitionTargets
- the partitionTargets to set
-
getRepartitioning
public int getRepartitioning()
- Returns:
- the repartitioning type
-
setRepartitioning
public void setRepartitioning(int repartitioning)
- Specified by:
setRepartitioning
in interfaceStepInterface
- Parameters:
repartitioning
- the repartitioning type to set
-
isPartitioned
public boolean isPartitioned()
- Specified by:
isPartitioned
in interfaceStepInterface
- Returns:
- the partitioned
-
setPartitioned
public void setPartitioned(boolean partitioned)
- Specified by:
setPartitioned
in interfaceStepInterface
- Parameters:
partitioned
- the partitioned to set
-
checkFeedback
protected boolean checkFeedback(long lines)
Check feedback.- Parameters:
lines
- the lines- Returns:
- true, if successful
-
getInputRowMeta
public org.pentaho.di.core.row.RowMetaInterface getInputRowMeta()
- Returns:
- the rowMeta
-
setInputRowMeta
public void setInputRowMeta(org.pentaho.di.core.row.RowMetaInterface rowMeta)
- Parameters:
rowMeta
- the rowMeta to set
-
getErrorRowMeta
public org.pentaho.di.core.row.RowMetaInterface getErrorRowMeta()
- Returns:
- the errorRowMeta
-
setErrorRowMeta
public void setErrorRowMeta(org.pentaho.di.core.row.RowMetaInterface errorRowMeta)
- Parameters:
errorRowMeta
- the errorRowMeta to set
-
getPreviewRowMeta
public org.pentaho.di.core.row.RowMetaInterface getPreviewRowMeta()
- Returns:
- the previewRowMeta
-
setPreviewRowMeta
public void setPreviewRowMeta(org.pentaho.di.core.row.RowMetaInterface previewRowMeta)
- Parameters:
previewRowMeta
- the previewRowMeta to set
-
copyVariablesFrom
public void copyVariablesFrom(org.pentaho.di.core.variables.VariableSpace space)
- Specified by:
copyVariablesFrom
in interfaceorg.pentaho.di.core.variables.VariableSpace
-
environmentSubstitute
public String environmentSubstitute(String aString)
- Specified by:
environmentSubstitute
in interfaceorg.pentaho.di.core.variables.VariableSpace
-
environmentSubstitute
public String environmentSubstitute(String aString, boolean escapeHexDelimiter)
- Specified by:
environmentSubstitute
in interfaceorg.pentaho.di.core.variables.VariableSpace
-
environmentSubstitute
public String[] environmentSubstitute(String[] aString)
- Specified by:
environmentSubstitute
in interfaceorg.pentaho.di.core.variables.VariableSpace
-
fieldSubstitute
public String fieldSubstitute(String aString, org.pentaho.di.core.row.RowMetaInterface rowMeta, Object[] rowData) throws org.pentaho.di.core.exception.KettleValueException
- Specified by:
fieldSubstitute
in interfaceorg.pentaho.di.core.variables.VariableSpace
- Throws:
org.pentaho.di.core.exception.KettleValueException
-
getParentVariableSpace
public org.pentaho.di.core.variables.VariableSpace getParentVariableSpace()
- Specified by:
getParentVariableSpace
in interfaceorg.pentaho.di.core.variables.VariableSpace
-
setParentVariableSpace
public void setParentVariableSpace(org.pentaho.di.core.variables.VariableSpace parent)
- Specified by:
setParentVariableSpace
in interfaceorg.pentaho.di.core.variables.VariableSpace
-
getVariable
public String getVariable(String variableName, String defaultValue)
- Specified by:
getVariable
in interfaceorg.pentaho.di.core.variables.VariableSpace
-
getVariable
public String getVariable(String variableName)
- Specified by:
getVariable
in interfaceorg.pentaho.di.core.variables.VariableSpace
-
getBooleanValueOfVariable
public boolean getBooleanValueOfVariable(String variableName, boolean defaultValue)
- Specified by:
getBooleanValueOfVariable
in interfaceorg.pentaho.di.core.variables.VariableSpace
-
initializeVariablesFrom
public void initializeVariablesFrom(org.pentaho.di.core.variables.VariableSpace parent)
- Specified by:
initializeVariablesFrom
in interfaceorg.pentaho.di.core.variables.VariableSpace
-
listVariables
public String[] listVariables()
- Specified by:
listVariables
in interfaceorg.pentaho.di.core.variables.VariableSpace
-
setVariable
public void setVariable(String variableName, String variableValue)
- Specified by:
setVariable
in interfaceorg.pentaho.di.core.variables.VariableSpace
-
shareVariablesWith
public void shareVariablesWith(org.pentaho.di.core.variables.VariableSpace space)
- Specified by:
shareVariablesWith
in interfaceorg.pentaho.di.core.variables.VariableSpace
-
injectVariables
public void injectVariables(Map<String,String> prop)
- Specified by:
injectVariables
in interfaceorg.pentaho.di.core.variables.VariableSpace
-
getTypeId
public String getTypeId()
Returns the step ID via the getStepID() method call. Support for CheckResultSourceInterface.- Returns:
- getStepID()
-
getSlaveNr
public int getSlaveNr()
Returns the unique slave number in the cluster.- Returns:
- the unique slave number in the cluster
-
getClusterSize
public int getClusterSize()
Returns the cluster size.- Returns:
- the cluster size
-
getUniqueStepNrAcrossSlaves
public int getUniqueStepNrAcrossSlaves()
Returns a unique step number across all slave servers: slaveNr * nrCopies + copyNr.- Returns:
- a unique step number across all slave servers: slaveNr * nrCopies + copyNr
-
getUniqueStepCountAcrossSlaves
public int getUniqueStepCountAcrossSlaves()
Returns the number of unique steps across all slave servers.- Returns:
- the number of unique steps across all slave servers
-
getServerSockets
public List<ServerSocket> getServerSockets()
Returns the serverSockets.- Returns:
- the serverSockets
-
setServerSockets
public void setServerSockets(List<ServerSocket> serverSockets)
- Parameters:
serverSockets
- the serverSockets to set
-
setUsingThreadPriorityManagment
public void setUsingThreadPriorityManagment(boolean usingThreadPriorityManagment)
Set to true to actively manage priorities of step threads.- Specified by:
setUsingThreadPriorityManagment
in interfaceStepInterface
- Parameters:
usingThreadPriorityManagment
- set to true to actively manage priorities of step threads
-
isUsingThreadPriorityManagment
public boolean isUsingThreadPriorityManagment()
Retusn true if we are actively managing priorities of step threads.- Specified by:
isUsingThreadPriorityManagment
in interfaceStepInterface
- Returns:
- true if we are actively managing priorities of step threads
-
initBeforeStart
public void initBeforeStart() throws org.pentaho.di.core.exception.KettleStepException
This method is executed by Trans right before the threads start and right after initialization.More to the point: here we open remote output step sockets.
- Specified by:
initBeforeStart
in interfaceStepInterface
- Throws:
org.pentaho.di.core.exception.KettleStepException
- In case there is an error
-
getStepListeners
public List<StepListener> getStepListeners()
Returns the step listeners.- Returns:
- the stepListeners
-
setStepListeners
public void setStepListeners(List<StepListener> stepListeners)
Sets the step listeners.- Parameters:
stepListeners
- the stepListeners to set
-
processRow
public boolean processRow(StepMetaInterface smi, StepDataInterface sdi) throws org.pentaho.di.core.exception.KettleException
Description copied from interface:StepInterface
Perform the equivalent of processing one row. Typically this means reading a row from input (getRow()) and passing a row to output (putRow)).- Specified by:
processRow
in interfaceStepInterface
- Parameters:
smi
- The steps metadata to work withsdi
- The steps temporary working data to work with (database connections, result sets, caches, temporary variables, etc.)- Returns:
- false if no more rows can be processed or an error occurred.
- Throws:
org.pentaho.di.core.exception.KettleException
-
beforeStartProcessing
public boolean beforeStartProcessing(StepMetaInterface smi, StepDataInterface sdi) throws org.pentaho.di.core.exception.KettleException
Description copied from interface:StepInterface
This method is executed by Trans right before starting processing rows.- Specified by:
beforeStartProcessing
in interfaceStepInterface
- Parameters:
smi
- The steps metadata to work withsdi
- The steps temporary working data to work with (database connections, result sets, caches, temporary variables, etc.)- Throws:
org.pentaho.di.core.exception.KettleException
-
canProcessOneRow
public boolean canProcessOneRow()
Description copied from interface:StepInterface
This method checks if the step is capable of processing at least one row.For example, if a step has no input records but needs at least one to function, it will return false.
- Specified by:
canProcessOneRow
in interfaceStepInterface
- Returns:
- true if the step can process a row.
-
addStepListener
public void addStepListener(StepListener stepListener)
Description copied from interface:StepInterface
Attach a step listener to be notified when a step arrives in a certain state. (finished)- Specified by:
addStepListener
in interfaceStepInterface
- Parameters:
stepListener
- The listener to add to the step
-
isMapping
public boolean isMapping()
- Specified by:
isMapping
in interfaceStepInterface
- Returns:
- true if the thread is a special mapping step
-
getSocketRepository
public SocketRepository getSocketRepository()
Retutns the socket repository.- Returns:
- the socketRepository
-
setSocketRepository
public void setSocketRepository(SocketRepository socketRepository)
Sets the socket repository.- Parameters:
socketRepository
- the socketRepository to set
-
getObjectName
public String getObjectName()
- Specified by:
getObjectName
in interfaceorg.pentaho.di.core.logging.LoggingObjectInterface
-
getLogChannel
public org.pentaho.di.core.logging.LogChannelInterface getLogChannel()
- Specified by:
getLogChannel
in interfaceHasLogChannelInterface
- Specified by:
getLogChannel
in interfaceStepInterface
- Returns:
- the logging channel for this step
-
getFilename
public String getFilename()
- Specified by:
getFilename
in interfaceorg.pentaho.di.core.logging.LoggingObjectInterface
-
getLogChannelId
public String getLogChannelId()
- Specified by:
getLogChannelId
in interfaceorg.pentaho.di.core.logging.LoggingObjectInterface
-
getObjectId
public org.pentaho.di.repository.ObjectId getObjectId()
- Specified by:
getObjectId
in interfaceorg.pentaho.di.core.logging.LoggingObjectInterface
-
getObjectRevision
public org.pentaho.di.repository.ObjectRevision getObjectRevision()
- Specified by:
getObjectRevision
in interfaceorg.pentaho.di.core.logging.LoggingObjectInterface
-
getObjectType
public org.pentaho.di.core.logging.LoggingObjectType getObjectType()
- Specified by:
getObjectType
in interfaceorg.pentaho.di.core.logging.LoggingObjectInterface
-
getParent
public org.pentaho.di.core.logging.LoggingObjectInterface getParent()
- Specified by:
getParent
in interfaceorg.pentaho.di.core.logging.LoggingObjectInterface
-
getRepositoryDirectory
public org.pentaho.di.repository.RepositoryDirectory getRepositoryDirectory()
- Specified by:
getRepositoryDirectory
in interfaceorg.pentaho.di.core.logging.LoggingObjectInterface
-
getObjectCopy
public String getObjectCopy()
- Specified by:
getObjectCopy
in interfaceorg.pentaho.di.core.logging.LoggingObjectInterface
-
getLogLevel
public org.pentaho.di.core.logging.LogLevel getLogLevel()
- Specified by:
getLogLevel
in interfaceorg.pentaho.di.core.logging.LoggingObjectInterface
-
setLogLevel
public void setLogLevel(org.pentaho.di.core.logging.LogLevel logLevel)
Sets the log level.- Parameters:
logLevel
- the new log level
-
closeQuietly
public static void closeQuietly(Closeable cl)
Close quietly.- Parameters:
cl
- the object that can be closed.
-
getContainerObjectId
public String getContainerObjectId()
Returns the container object ID.- Specified by:
getContainerObjectId
in interfaceorg.pentaho.di.core.logging.LoggingObjectInterface
- Returns:
- the containerObjectId
-
setCarteObjectId
public void setCarteObjectId(String containerObjectId)
Sets the container object ID.- Parameters:
containerObjectId
- the containerObjectId to set
-
batchComplete
public void batchComplete() throws org.pentaho.di.core.exception.KettleException
Description copied from interface:StepInterface
Calling this method will alert the step that we finished passing a batch of records to the step. Specifically for steps like "Sort Rows" it means that the buffered rows can be sorted and passed on.- Specified by:
batchComplete
in interfaceStepInterface
- Throws:
org.pentaho.di.core.exception.KettleException
- In case an error occurs during the processing of the batch of rows.
-
getRemoteInputSteps
public List<RemoteStep> getRemoteInputSteps()
Gets the remote input steps.- Returns:
- the remote input steps
-
getRemoteOutputSteps
public List<RemoteStep> getRemoteOutputSteps()
Gets the remote output steps.- Returns:
- the remote output steps
-
getRegistrationDate
public Date getRegistrationDate()
Returns the registration date- Specified by:
getRegistrationDate
in interfaceorg.pentaho.di.core.logging.LoggingObjectInterface
-
isGatheringMetrics
public boolean isGatheringMetrics()
- Specified by:
isGatheringMetrics
in interfaceorg.pentaho.di.core.logging.LoggingObjectInterface
-
setGatheringMetrics
public void setGatheringMetrics(boolean gatheringMetrics)
- Specified by:
setGatheringMetrics
in interfaceorg.pentaho.di.core.logging.LoggingObjectInterface
-
isForcingSeparateLogging
public boolean isForcingSeparateLogging()
- Specified by:
isForcingSeparateLogging
in interfaceorg.pentaho.di.core.logging.LoggingObjectInterface
-
isLoggingObjectInUse
public boolean isLoggingObjectInUse()
- Specified by:
isLoggingObjectInUse
in interfaceorg.pentaho.di.core.logging.LoggingObjectInterface
-
setLoggingObjectInUse
public void setLoggingObjectInUse(boolean inUse)
-
setForcingSeparateLogging
public void setForcingSeparateLogging(boolean forcingSeparateLogging)
- Specified by:
setForcingSeparateLogging
in interfaceorg.pentaho.di.core.logging.LoggingObjectInterface
-
getRepository
public Repository getRepository()
- Specified by:
getRepository
in interfaceStepInterface
- Returns:
- The repository used by the step to load and reference Kettle objects with at runtime
-
setRepository
public void setRepository(Repository repository)
- Specified by:
setRepository
in interfaceStepInterface
- Parameters:
repository
- The repository used by the step to load and reference Kettle objects with at runtime
-
getMetaStore
public org.pentaho.metastore.api.IMetaStore getMetaStore()
- Specified by:
getMetaStore
in interfaceStepInterface
- Returns:
- The metastore that the step uses to load external elements from.
-
setMetaStore
public void setMetaStore(org.pentaho.metastore.api.IMetaStore metaStore)
Description copied from interface:StepInterface
Pass along the metastore to use when loading external elements at runtime.- Specified by:
setMetaStore
in interfaceStepInterface
- Parameters:
metaStore
- The metastore to use
-
getCurrentOutputRowSetNr
public int getCurrentOutputRowSetNr()
- Specified by:
getCurrentOutputRowSetNr
in interfaceStepInterface
- Returns:
- the index of the active (current) output row set
-
setCurrentOutputRowSetNr
public void setCurrentOutputRowSetNr(int index)
- Specified by:
setCurrentOutputRowSetNr
in interfaceStepInterface
- Parameters:
index
- Sets the index of the active (current) output row set to use.
-
getCurrentInputRowSetNr
public int getCurrentInputRowSetNr()
- Specified by:
getCurrentInputRowSetNr
in interfaceStepInterface
- Returns:
- the index of the active (current) input row set
-
setCurrentInputRowSetNr
public void setCurrentInputRowSetNr(int index)
- Specified by:
setCurrentInputRowSetNr
in interfaceStepInterface
- Parameters:
index
- Sets the index of the active (current) input row set to use.
-
-