org.pentaho.di.trans.steps.monetdbbulkloader

Class MonetDBBulkLoader

java.lang.Object

org.pentaho.di.trans.step.BaseStep

org.pentaho.di.trans.steps.monetdbbulkloader.MonetDBBulkLoader

All Implemented Interfaces:

ExtensionDataInterface, HasLogChannelInterface, LoggingObjectInterface, VariableSpace, StepInterface

Direct Known Subclasses:

MonetDBAgileMart

public class MonetDBBulkLoader
extends BaseStep
implements StepInterface

Performs a bulk load to a MonetDB table.

Based on (copied from) Sven Boden's Oracle Bulk Loader step

Since:

22-aug-2008

Author:

matt

Field Summary

Fields

Modifier and Type	Field and Description
protected long	rowsWritten

Fields inherited from class org.pentaho.di.trans.step.BaseStep

deadLockCounter, extensionDataMap, first, linesInput, linesOutput, linesRead, linesRejected, linesSkipped, linesUpdated, linesWritten, log, metaStore, repository, rowListeners, safeStopped, terminator, terminator_rows, variables

Constructor Summary

Constructors

Constructor and Description
MonetDBBulkLoader(StepMeta stepMeta, StepDataInterface stepDataInterface, int copyNr, TransMeta transMeta, Trans trans)

Method Summary

Modifier and Type	Method and Description
protected void	addRowToBuffer(RowMetaInterface rowMeta, Object[] r)
void	autoAdjustSchema(MonetDBBulkLoaderMeta meta)
void	dispose(StepMetaInterface smi, StepDataInterface sdi) Dispose of this step: close files, empty logs, etc.
void	drop()
protected String	escapeOsPath(String path, boolean isWindows)
boolean	execute(MonetDBBulkLoaderMeta meta, boolean wait)
protected void	executeSql(String query)
protected static void	executeSql(String query, String host, int port, String user, String password, String db)
protected MonetDBBulkLoaderData	getData()
String	getMessage()
protected MonetDBBulkLoaderMeta	getMeta()
protected nl.cwi.monetdb.mcl.net.MapiSocket	getMonetDBConnection()
protected static nl.cwi.monetdb.mcl.net.MapiSocket	getMonetDBConnection(String host, int port, String user, String password, String db)
protected static nl.cwi.monetdb.mcl.net.MapiSocket	getMonetDBConnection(String host, int port, String user, String password, String db, LogChannelInterface log)
boolean	init(StepMetaInterface smi, StepDataInterface sdi) Initialize and do work where other steps need to wait for...
boolean	processRow(StepMetaInterface smi, StepDataInterface sdi) Perform the equivalent of processing one row.
protected void	setMessage(String message)
void	truncate()
protected void	verifyDatabaseConnection()
protected void	writeBufferToMonetDB()
protected void	writeRowToMonetDB(RowMetaInterface rowMeta, Object[] r)

Methods inherited from class org.pentaho.di.trans.step.BaseStep

addResultFile, addRowListener, addRowSetToInputRowSets, addRowSetToOutputRowSets, addStepListener, batchComplete, buildLog, canProcessOneRow, checkFeedback, cleanup, clearInputRowSets, clearOutputRowSets, closeQuietly, copyVariablesFrom, decrementLinesRead, decrementLinesWritten, dispatch, environmentSubstitute, environmentSubstitute, fieldSubstitute, findInputRowSet, findInputRowSet, findOutputRowSet, findOutputRowSet, getBooleanValueOfVariable, getClusterSize, getContainerObjectId, getCopy, getCurrentInputRowSetNr, getCurrentOutputRowSetNr, getDispatcher, getErrorRowMeta, getErrors, getExtensionDataMap, getFilename, getFirstInputRowSet, getInputRowMeta, getInputRowSets, getLinesInput, getLinesOutput, getLinesRead, getLinesRejected, getLinesSkipped, getLinesUpdated, getLinesWritten, getLogChannel, getLogChannelId, getLogFields, getLogLevel, getMetaStore, getNextClassNr, getObjectCopy, getObjectId, getObjectName, getObjectRevision, getObjectType, getOutputRowSets, getParent, getParentVariableSpace, getPartitionID, getPartitionTargets, getPreviewRowMeta, getProcessed, getRegistrationDate, getRemoteInputSteps, getRemoteOutputSteps, getRepartitioning, getRepository, getRepositoryDirectory, getResultFiles, getRow, getRowFrom, getRowHandler, getRowListeners, getRuntime, getServerSockets, getSlaveNr, getSocketRepository, getStatus, getStatusDescription, getStepDataInterface, getStepID, getStepListeners, getStepMeta, getStepMetaInterface, getStepname, getTrans, getTransMeta, getTypeId, getUniqueStepCountAcrossSlaves, getUniqueStepNrAcrossSlaves, getVariable, getVariable, handleGetRowFrom, handlePutRowTo, identifyErrorOutput, incrementLinesInput, incrementLinesOutput, incrementLinesRead, incrementLinesRejected, incrementLinesSkipped, incrementLinesUpdated, incrementLinesWritten, initBeforeStart, initializeVariablesFrom, injectVariables, isBasic, isDebug, isDetailed, isDistributed, isForcingSeparateLogging, isGatheringMetrics, isInitialising, isMapping, isPartitioned, isPaused, isRowLevel, isRunning, isSafeStopped, isStopped, isUsingThreadPriorityManagment, listVariables, logBasic, logBasic, logDebug, logDebug, logDetailed, logDetailed, logError, logError, logError, logMinimal, logMinimal, logRowlevel, logRowlevel, logSummary, markStart, markStop, openRemoteInputStepSocketsOnce, openRemoteOutputStepSocketsOnce, outputIsDone, pauseRunning, putError, putRow, putRowTo, removeRowListener, resumeRunning, rowsetInputSize, rowsetOutputSize, safeModeChecking, safeModeChecking, setCarteObjectId, setCopy, setCurrentInputRowSetNr, setCurrentOutputRowSetNr, setDistributed, setErrorRowMeta, setErrors, setForcingSeparateLogging, setGatheringMetrics, setInputRowMeta, setInputRowSets, setInternalVariables, setLinesInput, setLinesOutput, setLinesRead, setLinesRejected, setLinesSkipped, setLinesUpdated, setLinesWritten, setLogLevel, setMetaStore, setOutputDone, setOutputRowSets, setParentVariableSpace, setPartitioned, setPartitionID, setPartitionTargets, setPaused, setPaused, setPreviewRowMeta, setRepartitioning, setRepository, setRowHandler, setRunning, setSafeStopped, setServerSockets, setSocketRepository, setStepDataInterface, setStepListeners, setStepMeta, setStepMetaInterface, setStepname, setStopped, setTransMeta, setUsingThreadPriorityManagment, setVariable, shareVariablesWith, stopAll, stopRunning, stopRunning, swapFirstInputRowSetIfExists, toString, verifyInputDeadLock, waitUntilTransformationIsStarted

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.pentaho.di.trans.step.StepInterface

addRowListener, addRowSetToInputRowSets, addRowSetToOutputRowSets, addStepListener, batchComplete, canProcessOneRow, cleanup, getCopy, getCurrentInputRowSetNr, getCurrentOutputRowSetNr, getErrors, getInputRowSets, getLinesInput, getLinesOutput, getLinesRead, getLinesRejected, getLinesUpdated, getLinesWritten, getLogChannel, getMetaStore, getOutputRowSets, getPartitionID, getProcessed, getRepository, getResultFiles, getRow, getRowListeners, getRuntime, getStatus, getStepID, getStepMeta, getStepname, getTrans, identifyErrorOutput, initBeforeStart, isMapping, isPartitioned, isPaused, isRunning, isSafeStopped, isStopped, isUsingThreadPriorityManagment, markStart, markStop, pauseRunning, putRow, removeRowListener, resumeRunning, rowsetInputSize, rowsetOutputSize, setCurrentInputRowSetNr, setCurrentOutputRowSetNr, setErrors, setLinesRejected, setMetaStore, setOutputDone, setPartitioned, setPartitionID, setRepartitioning, setRepository, setRunning, setSafeStopped, setStopped, setUsingThreadPriorityManagment, stopAll, stopRunning, subStatuses

Methods inherited from interface org.pentaho.di.core.variables.VariableSpace

copyVariablesFrom, environmentSubstitute, environmentSubstitute, fieldSubstitute, getBooleanValueOfVariable, getParentVariableSpace, getVariable, getVariable, initializeVariablesFrom, injectVariables, listVariables, setParentVariableSpace, setVariable, shareVariablesWith

Field Detail

rowsWritten

protected long rowsWritten

Constructor Detail

MonetDBBulkLoader

public MonetDBBulkLoader(StepMeta stepMeta,
                         StepDataInterface stepDataInterface,
                         int copyNr,
                         TransMeta transMeta,
                         Trans trans)

Method Detail

getMessage

public String getMessage()

setMessage

protected void setMessage(String message)

getMeta

protected MonetDBBulkLoaderMeta getMeta()

escapeOsPath

protected String escapeOsPath(String path,
                              boolean isWindows)

execute

public boolean execute(MonetDBBulkLoaderMeta meta,
                       boolean wait)
                throws KettleException

Throws:

KettleException

processRow

public boolean processRow(StepMetaInterface smi,
                          StepDataInterface sdi)
                   throws KettleException

Description copied from interface: StepInterface

Perform the equivalent of processing one row. Typically this means reading a row from input (getRow()) and passing a row to output (putRow)).

Specified by:

processRow in interface StepInterface

Overrides:

processRow in class BaseStep

Parameters:

smi - The steps metadata to work with

sdi - The steps temporary working data to work with (database connections, result sets, caches, temporary variables, etc.)

Returns:

false if no more rows can be processed or an error occurred.

Throws:

KettleException

writeRowToMonetDB

protected void writeRowToMonetDB(RowMetaInterface rowMeta,
                                 Object[] r)
                          throws KettleException

Throws:

KettleException

addRowToBuffer

protected void addRowToBuffer(RowMetaInterface rowMeta,
                              Object[] r)
                       throws KettleException

Throws:

KettleException

truncate

public void truncate()
              throws KettleException

Throws:

KettleException

drop

public void drop()
          throws KettleException

Throws:

KettleException

autoAdjustSchema

public void autoAdjustSchema(MonetDBBulkLoaderMeta meta)
                      throws KettleException

Throws:

KettleException

writeBufferToMonetDB

protected void writeBufferToMonetDB()
                             throws KettleException

Throws:

KettleException

verifyDatabaseConnection

protected void verifyDatabaseConnection()
                                 throws KettleException

Throws:

KettleException

init

public boolean init(StepMetaInterface smi,
                    StepDataInterface sdi)

Description copied from interface: StepInterface

Initialize and do work where other steps need to wait for...

Specified by:

init in interface StepInterface

Overrides:

init in class BaseStep

Parameters:

smi - The metadata to work with

sdi - The data to initialize

dispose

public void dispose(StepMetaInterface smi,
                    StepDataInterface sdi)

Description copied from interface: StepInterface

Dispose of this step: close files, empty logs, etc.

Specified by:

dispose in interface StepInterface

Overrides:

dispose in class BaseStep

Parameters:

smi - The metadata to work with

sdi - The data to dispose of

getData

protected MonetDBBulkLoaderData getData()

getMonetDBConnection

protected nl.cwi.monetdb.mcl.net.MapiSocket getMonetDBConnection()
                                                          throws Exception

Throws:

Exception

getMonetDBConnection

protected static nl.cwi.monetdb.mcl.net.MapiSocket getMonetDBConnection(String host,
                                                                        int port,
                                                                        String user,
                                                                        String password,
                                                                        String db)
                                                                 throws Exception

Throws:

Exception

getMonetDBConnection

protected static nl.cwi.monetdb.mcl.net.MapiSocket getMonetDBConnection(String host,
                                                                        int port,
                                                                        String user,
                                                                        String password,
                                                                        String db,
                                                                        LogChannelInterface log)
                                                                 throws Exception

Throws:

Exception

executeSql

protected void executeSql(String query)
                   throws Exception

Throws:

Exception

executeSql

protected static void executeSql(String query,
                                 String host,
                                 int port,
                                 String user,
                                 String password,
                                 String db)
                          throws Exception

Throws:

Exception