org.pentaho.di.trans.steps.parallelgzipcsv

Class ParGzipCsvInputMeta

java.lang.Object

org.pentaho.di.trans.step.BaseStepMeta

org.pentaho.di.trans.steps.parallelgzipcsv.ParGzipCsvInputMeta

All Implemented Interfaces:

Cloneable, StepAttributesInterface, StepMetaInterface, InputFileMetaInterface

public class ParGzipCsvInputMeta
extends BaseStepMeta
implements StepMetaInterface, InputFileMetaInterface

Since:

2009-03-06

Version:

3.2

Author:

matt

Field Summary

Fields inherited from class org.pentaho.di.trans.step.BaseStepMeta

loggingObject, STEP_ATTRIBUTES_FILE

Constructor Summary

Constructors

Constructor and Description
ParGzipCsvInputMeta()

Method Summary

Modifier and Type	Method and Description
void	allocate(int nrFields)
void	check(List<org.pentaho.di.core.CheckResultInterface> remarks, TransMeta transMeta, StepMeta stepMeta, org.pentaho.di.core.row.RowMetaInterface prev, String[] input, String[] output, org.pentaho.di.core.row.RowMetaInterface info, org.pentaho.di.core.variables.VariableSpace space, Repository repository, org.pentaho.metastore.api.IMetaStore metaStore) Checks the settings of this step and puts the findings in a remarks List.
Object	clone() Make an exact copy of this step, make sure to explicitly copy Collections etc.
String	exportResources(org.pentaho.di.core.variables.VariableSpace space, Map<String,ResourceDefinition> definitions, ResourceNamingInterface resourceNamingInterface, Repository repository, org.pentaho.metastore.api.IMetaStore metaStore) Since the exported transformation that runs this will reside in a ZIP file, we can't reference files relatively.
String	getBufferSize()
String	getDelimiter()
String	getEnclosure()
String	getEncoding()
String	getErrorCountField()
String	getErrorFieldsField()
String	getErrorTextField()
String	getEscapeCharacter()
void	getFields(org.pentaho.di.core.row.RowMetaInterface rowMeta, String origin, org.pentaho.di.core.row.RowMetaInterface[] info, StepMeta nextStep, org.pentaho.di.core.variables.VariableSpace space, Repository repository, org.pentaho.metastore.api.IMetaStore metaStore) Gets the fields.
int	getFileFormatTypeNr()
String	getFilename()
String	getFilenameField()
String[]	getFilePaths(org.pentaho.di.core.variables.VariableSpace space)
String	getFileType()
TextFileInputField[]	getInputFields()
int	getNrHeaderLines()
List<ResourceReference>	getResourceDependencies(TransMeta transMeta, StepMeta stepInfo) Get a list of all the resource dependencies that the step is depending on.
String	getRowNumField()
String	getSeparator()
StepInterface	getStep(StepMeta stepMeta, StepDataInterface stepDataInterface, int cnr, TransMeta tr, Trans trans) Get the executing step, needed by Trans to launch a step.
StepDataInterface	getStepData() Get a new instance of the appropriate data class.
StepMetaInjectionInterface	getStepMetaInjectionInterface()
String	getXML() Produces the XML string that describes this step's information.
boolean	hasHeader()
boolean	includeFilename()
boolean	includeRowNumber()
boolean	isAddResultFile()
boolean	isErrorIgnored()
boolean	isErrorLineSkipped()
boolean	isHeaderPresent()
boolean	isIncludingFilename()
boolean	isLazyConversionActive()
boolean	isRunningInParallel()
void	loadXML(Node stepnode, List<org.pentaho.di.core.database.DatabaseMeta> databases, org.pentaho.metastore.api.IMetaStore metaStore) Load the values for this step from an XML Node
void	readRep(Repository rep, org.pentaho.metastore.api.IMetaStore metaStore, org.pentaho.di.repository.ObjectId id_step, List<org.pentaho.di.core.database.DatabaseMeta> databases) Read the steps information from a Kettle repository
void	saveRep(Repository rep, org.pentaho.metastore.api.IMetaStore metaStore, org.pentaho.di.repository.ObjectId id_transformation, org.pentaho.di.repository.ObjectId id_step) Save the steps data into a Kettle repository
void	setAddResultFile(boolean isaddresult)
void	setBufferSize(String bufferSize)
void	setDefault() Set default values
void	setDelimiter(String delimiter)
void	setEnclosure(String enclosure)
void	setEncoding(String encoding)
void	setFilename(String filename)
void	setFilenameField(String filenameField)
void	setHeaderPresent(boolean headerPresent)
void	setIncludingFilename(boolean includingFilename)
void	setInputFields(TextFileInputField[] inputFields)
void	setLazyConversionActive(boolean lazyConversionActive)
void	setRowNumField(String rowNumField)
void	setRunningInParallel(boolean runningInParallel)

Methods inherited from class org.pentaho.di.trans.step.BaseStepMeta

analyseImpact, analyseImpact, cancelQueries, check, check, excludeFromCopyDistributeVerification, excludeFromRowLayoutVerification, exportResources, extractStepMetadataEntries, findAttribute, findParent, getActiveReferencedObjectDescription, getDescription, getDialogClassName, getFields, getLog, getLogChannelId, getName, getObjectCopy, getObjectId, getObjectRevision, getObjectType, getOptionalStreams, getParent, getParentStepMeta, getReferencedObjectDescriptions, getRepCode, getRepositoryDirectory, getRequiredFields, getRequiredFields, getSQLStatements, getSQLStatements, getStepInjectionMetadataEntries, getStepIOMeta, getSupportedTransformationTypes, getTableFields, getTooltip, getUsedArguments, getUsedDatabaseConnections, getUsedLibraries, getXmlCode, handleStreamSelection, hasChanged, hasRepositoryReferences, isBasic, isDebug, isDetailed, isReferencedObjectEnabled, isRowLevel, loadReferencedObject, loadReferencedObject, loadXML, loadXML, logBasic, logBasic, logDebug, logDebug, logDetailed, logDetailed, logError, logError, logError, logMinimal, logMinimal, logRowlevel, logRowlevel, lookupRepositoryReferences, readRep, readRep, resetStepIoMeta, saveRep, searchInfoAndTargetSteps, setChanged, setChanged, setParentStepMeta, supportsErrorHandling

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.pentaho.di.trans.step.StepMetaInterface

analyseImpact, analyseImpact, cancelQueries, check, excludeFromCopyDistributeVerification, excludeFromRowLayoutVerification, exportResources, extractStepMetadataEntries, getActiveReferencedObjectDescription, getDialogClassName, getFields, getOptionalStreams, getParentStepMeta, getReferencedObjectDescriptions, getRequiredFields, getSQLStatements, getSQLStatements, getStepIOMeta, getSupportedTransformationTypes, getTableFields, getUsedArguments, getUsedDatabaseConnections, getUsedLibraries, handleStreamSelection, hasChanged, hasRepositoryReferences, isReferencedObjectEnabled, loadReferencedObject, loadXML, lookupRepositoryReferences, readRep, resetStepIoMeta, saveRep, searchInfoAndTargetSteps, setChanged, setParentStepMeta, supportsErrorHandling

Constructor Detail

ParGzipCsvInputMeta

public ParGzipCsvInputMeta()

Method Detail

loadXML

public void loadXML(Node stepnode,
                    List<org.pentaho.di.core.database.DatabaseMeta> databases,
                    org.pentaho.metastore.api.IMetaStore metaStore)
             throws org.pentaho.di.core.exception.KettleXMLException

Description copied from interface: StepMetaInterface

Load the values for this step from an XML Node

Specified by:

loadXML in interface StepMetaInterface

Overrides:

loadXML in class BaseStepMeta

Parameters:

stepnode - the Node to get the info from

databases - The available list of databases to reference to

metaStore - the metastore to optionally load external reference metadata from

Throws:

org.pentaho.di.core.exception.KettleXMLException - When an unexpected XML error occurred. (malformed etc.)

clone

public Object clone()

Description copied from interface: StepMetaInterface

Make an exact copy of this step, make sure to explicitly copy Collections etc.

Specified by:

clone in interface StepMetaInterface

Overrides:

clone in class BaseStepMeta

Returns:

an exact copy of this step

setDefault

public void setDefault()

Description copied from interface: StepMetaInterface

Set default values

Specified by:

setDefault in interface StepMetaInterface

allocate

public void allocate(int nrFields)

getXML

public String getXML()

Description copied from class: BaseStepMeta

Produces the XML string that describes this step's information.

Specified by:

getXML in interface StepMetaInterface

Overrides:

getXML in class BaseStepMeta

Returns:

String containing the XML describing this step.

readRep

public void readRep(Repository rep,
                    org.pentaho.metastore.api.IMetaStore metaStore,
                    org.pentaho.di.repository.ObjectId id_step,
                    List<org.pentaho.di.core.database.DatabaseMeta> databases)
             throws org.pentaho.di.core.exception.KettleException

Description copied from interface: StepMetaInterface

Read the steps information from a Kettle repository

Specified by:

readRep in interface StepMetaInterface

Overrides:

readRep in class BaseStepMeta

Parameters:

rep - The repository to read from

metaStore - The MetaStore to read external information from

id_step - The step ID

databases - The databases to reference

Throws:

org.pentaho.di.core.exception.KettleException - When an unexpected error occurred (database, network, etc)

saveRep

public void saveRep(Repository rep,
                    org.pentaho.metastore.api.IMetaStore metaStore,
                    org.pentaho.di.repository.ObjectId id_transformation,
                    org.pentaho.di.repository.ObjectId id_step)
             throws org.pentaho.di.core.exception.KettleException

Description copied from interface: StepMetaInterface

Save the steps data into a Kettle repository

Specified by:

saveRep in interface StepMetaInterface

Overrides:

saveRep in class BaseStepMeta

Parameters:

rep - The Kettle repository to save to

metaStore - the metaStore to optionally write to

id_transformation - The transformation ID

id_step - The step ID

Throws:

org.pentaho.di.core.exception.KettleException - When an unexpected error occurred (database, network, etc)

getFields

public void getFields(org.pentaho.di.core.row.RowMetaInterface rowMeta,
                      String origin,
                      org.pentaho.di.core.row.RowMetaInterface[] info,
                      StepMeta nextStep,
                      org.pentaho.di.core.variables.VariableSpace space,
                      Repository repository,
                      org.pentaho.metastore.api.IMetaStore metaStore)
               throws org.pentaho.di.core.exception.KettleStepException

Description copied from class: BaseStepMeta

Gets the fields.

Specified by:

getFields in interface StepMetaInterface

Overrides:

getFields in class BaseStepMeta

Parameters:

rowMeta - the input row meta that is modified in this method to reflect the output row metadata of the step

origin - Name of the step to use as input for the origin field in the values

info - Fields used as extra lookup information

nextStep - the next step that is targeted

space - the space The variable space to use to replace variables

repository - the repository to use to load Kettle metadata objects impacting the output fields

metaStore - the MetaStore to use to load additional external data or metadata impacting the output fields

Throws:

org.pentaho.di.core.exception.KettleStepException - the kettle step exception

check

public void check(List<org.pentaho.di.core.CheckResultInterface> remarks,
                  TransMeta transMeta,
                  StepMeta stepMeta,
                  org.pentaho.di.core.row.RowMetaInterface prev,
                  String[] input,
                  String[] output,
                  org.pentaho.di.core.row.RowMetaInterface info,
                  org.pentaho.di.core.variables.VariableSpace space,
                  Repository repository,
                  org.pentaho.metastore.api.IMetaStore metaStore)

Description copied from interface: StepMetaInterface

Checks the settings of this step and puts the findings in a remarks List.

Specified by:

check in interface StepMetaInterface

Overrides:

check in class BaseStepMeta

Parameters:

remarks - The list to put the remarks in @see org.pentaho.di.core.CheckResult

stepMeta - The stepMeta to help checking

prev - The fields coming from the previous step

input - The input step names

output - The output step names

info - The fields that are used as information by the step

space - the variable space to resolve variable expressions with

repository - the repository to use to load Kettle metadata objects impacting the output fields

metaStore - the MetaStore to use to load additional external data or metadata impacting the output fields

getStep

public StepInterface getStep(StepMeta stepMeta,
                             StepDataInterface stepDataInterface,
                             int cnr,
                             TransMeta tr,
                             Trans trans)

Description copied from interface: StepMetaInterface

Get the executing step, needed by Trans to launch a step.

Specified by:

getStep in interface StepMetaInterface

Parameters:

stepMeta - The step info

stepDataInterface - the step data interface linked to this step. Here the step can store temporary data, database connections, etc.

cnr - The copy nr to get

tr - The transformation info

trans - The launching transformation

getStepData

public StepDataInterface getStepData()

Description copied from interface: StepMetaInterface

Get a new instance of the appropriate data class. This data class implements the StepDataInterface. It basically contains the persisting data that needs to live on, even if a worker thread is terminated.

Specified by:

getStepData in interface StepMetaInterface

Returns:

The appropriate StepDataInterface class.

getDelimiter

public String getDelimiter()

Returns:

the delimiter

setDelimiter

public void setDelimiter(String delimiter)

Parameters:

delimiter - the delimiter to set

getFilename

public String getFilename()

Returns:

the filename

setFilename

public void setFilename(String filename)

Parameters:

filename - the filename to set

getBufferSize

public String getBufferSize()

Returns:

the bufferSize

setBufferSize

public void setBufferSize(String bufferSize)

Parameters:

bufferSize - the bufferSize to set

isLazyConversionActive

public boolean isLazyConversionActive()

Returns:

true if lazy conversion is turned on: conversions are delayed as long as possible, perhaps to never occur at all.

setLazyConversionActive

public void setLazyConversionActive(boolean lazyConversionActive)

Parameters:

lazyConversionActive - true if lazy conversion is to be turned on: conversions are delayed as long as possible, perhaps to never occur at all.

isHeaderPresent

public boolean isHeaderPresent()

Returns:

the headerPresent

setHeaderPresent

public void setHeaderPresent(boolean headerPresent)

Parameters:

headerPresent - the headerPresent to set

getEnclosure

public String getEnclosure()

Specified by:

getEnclosure in interface InputFileMetaInterface

Returns:

the enclosure

setEnclosure

public void setEnclosure(String enclosure)

Parameters:

enclosure - the enclosure to set

getResourceDependencies

public List<ResourceReference> getResourceDependencies(TransMeta transMeta,
                                                       StepMeta stepInfo)

Description copied from class: BaseStepMeta

Get a list of all the resource dependencies that the step is depending on.

Specified by:

getResourceDependencies in interface StepMetaInterface

Overrides:

getResourceDependencies in class BaseStepMeta

Returns:

a list of all the resource dependencies that the step is depending on

getInputFields

public TextFileInputField[] getInputFields()

Specified by:

getInputFields in interface InputFileMetaInterface

Returns:

the inputFields

setInputFields

public void setInputFields(TextFileInputField[] inputFields)

Parameters:

inputFields - the inputFields to set

getFileFormatTypeNr

public int getFileFormatTypeNr()

Specified by:

getFileFormatTypeNr in interface InputFileMetaInterface

getFilePaths

public String[] getFilePaths(org.pentaho.di.core.variables.VariableSpace space)

Specified by:

getFilePaths in interface InputFileMetaInterface

getNrHeaderLines

public int getNrHeaderLines()

Specified by:

getNrHeaderLines in interface InputFileMetaInterface

hasHeader

public boolean hasHeader()

Specified by:

hasHeader in interface InputFileMetaInterface

getErrorCountField

public String getErrorCountField()

Specified by:

getErrorCountField in interface InputFileMetaInterface

getErrorFieldsField

public String getErrorFieldsField()

Specified by:

getErrorFieldsField in interface InputFileMetaInterface

getErrorTextField

public String getErrorTextField()

Specified by:

getErrorTextField in interface InputFileMetaInterface

getEscapeCharacter

public String getEscapeCharacter()

Specified by:

getEscapeCharacter in interface InputFileMetaInterface

getFileType

public String getFileType()

Specified by:

getFileType in interface InputFileMetaInterface

getSeparator

public String getSeparator()

Specified by:

getSeparator in interface InputFileMetaInterface

includeFilename

public boolean includeFilename()

Specified by:

includeFilename in interface InputFileMetaInterface

includeRowNumber

public boolean includeRowNumber()

Specified by:

includeRowNumber in interface InputFileMetaInterface

isErrorIgnored

public boolean isErrorIgnored()

Specified by:

isErrorIgnored in interface InputFileMetaInterface

isErrorLineSkipped

public boolean isErrorLineSkipped()

Specified by:

isErrorLineSkipped in interface InputFileMetaInterface

getFilenameField

public String getFilenameField()

Returns:

the filenameField

setFilenameField

public void setFilenameField(String filenameField)

Parameters:

filenameField - the filenameField to set

isIncludingFilename

public boolean isIncludingFilename()

Returns:

the includingFilename

setIncludingFilename

public void setIncludingFilename(boolean includingFilename)

Parameters:

includingFilename - the includingFilename to set

getRowNumField

public String getRowNumField()

Returns:

the rowNumField

setRowNumField

public void setRowNumField(String rowNumField)

Parameters:

rowNumField - the rowNumField to set

setAddResultFile

public void setAddResultFile(boolean isaddresult)

Parameters:

isaddresult - The isaddresult to set.

isAddResultFile

public boolean isAddResultFile()

Returns:

Returns isaddresult.

isRunningInParallel

public boolean isRunningInParallel()

Returns:

the runningInParallel

setRunningInParallel

public void setRunningInParallel(boolean runningInParallel)

Parameters:

runningInParallel - the runningInParallel to set

getEncoding

public String getEncoding()

Returns:

the encoding

setEncoding

public void setEncoding(String encoding)

Parameters:

encoding - the encoding to set

exportResources

public String exportResources(org.pentaho.di.core.variables.VariableSpace space,
                              Map<String,ResourceDefinition> definitions,
                              ResourceNamingInterface resourceNamingInterface,
                              Repository repository,
                              org.pentaho.metastore.api.IMetaStore metaStore)
                       throws org.pentaho.di.core.exception.KettleException

Since the exported transformation that runs this will reside in a ZIP file, we can't reference files relatively. So what this does is turn the name of files into absolute paths OR it simply includes the resource in the ZIP file. For now, we'll simply turn it into an absolute path and pray that the file is on a shared drive or something like that.

Specified by:

exportResources in interface StepMetaInterface

Overrides:

exportResources in class BaseStepMeta

Parameters:

space - the variable space to use

definitions -

resourceNamingInterface -

repository - The repository to optionally load other resources from (to be converted to XML)

metaStore - the metaStore in which non-kettle metadata could reside.

Returns:

the filename of the exported resource

Throws:

org.pentaho.di.core.exception.KettleException

getStepMetaInjectionInterface

public StepMetaInjectionInterface getStepMetaInjectionInterface()

Specified by:

getStepMetaInjectionInterface in interface StepMetaInterface

Overrides:

getStepMetaInjectionInterface in class BaseStepMeta

Returns:

Optional interface that allows an external program to inject step metadata in a standardized fasion. This method will return null if the interface is not available for this step.