org.pentaho.di.trans.steps.fileinput.text

Class TextFileInputMeta

java.lang.Object

org.pentaho.di.trans.step.BaseStepMeta

org.pentaho.di.trans.steps.file.BaseFileMeta

org.pentaho.di.trans.steps.file.BaseFileInputMeta<BaseFileInputAdditionalField,BaseFileInputFiles,BaseFileField>

org.pentaho.di.trans.steps.fileinput.text.TextFileInputMeta

All Implemented Interfaces:

Cloneable, StepAttributesInterface, StepMetaInterface, ResolvableResource

public class TextFileInputMeta
extends BaseFileInputMeta<BaseFileInputAdditionalField,BaseFileInputFiles,BaseFileField>
implements StepMetaInterface, ResolvableResource

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	TextFileInputMeta.Content

Field Summary

Fields

Modifier and Type	Field and Description
TextFileInputMeta.Content	content
String	errorCountField The name of the field that will contain the number of errors in the row
String	errorFieldsField The name of the field that will contain the names of the fields that generated errors, separated by ,
boolean	errorLineSkipped If error line are skipped, you can replay without introducing doubles.
String	errorTextField The name of the field that will contain the error texts, separated by CR
static int	FILE_FORMAT_DOS
static int	FILE_FORMAT_MIXED
static int	FILE_FORMAT_UNIX
static int	FILE_TYPE_CSV
static int	FILE_TYPE_FIXED

Fields inherited from class org.pentaho.di.trans.steps.file.BaseFileInputMeta

additionalOutputFields, errorHandling, inputFields, inputFiles, NO, RequiredFilesCode, RequiredFilesDesc, YES

Fields inherited from class org.pentaho.di.trans.steps.file.BaseFileMeta

servletOutput

Fields inherited from class org.pentaho.di.trans.step.BaseStepMeta

attributes, databases, ioMeta, log, loggingObject, parentStepMeta, repository, STEP_ATTRIBUTES_FILE

Constructor Summary

Constructors

Constructor and Description
TextFileInputMeta()

Method Summary

Modifier and Type	Method and Description
void	allocate(int nrfiles, int nrfields, int nrfilters)
void	allocateFiles(int nrFiles)
void	check(List<CheckResultInterface> remarks, TransMeta transMeta, StepMeta stepMeta, RowMetaInterface prev, String[] input, String[] output, RowMetaInterface info, VariableSpace space, Repository repository, org.pentaho.metastore.api.IMetaStore metaStore) Checks the settings of this step and puts the findings in a remarks List.
Object	clone() Make an exact copy of this step, make sure to explicitly copy Collections etc.
String	exportResources(VariableSpace space, Map<String,ResourceDefinition> definitions, ResourceNamingInterface resourceNamingInterface, Repository repository, org.pentaho.metastore.api.IMetaStore metaStore) Since the exported transformation that runs this will reside in a ZIP file, we can't reference files relatively.
String	getAcceptingField() Required for the Data Lineage.
StepMeta	getAcceptingStep() Required for the Data Lineage.
String	getAcceptingStepName() Required for the Data Lineage.
String	getEncoding()
String	getErrorCountField()
String	getErrorFieldsField()
String	getErrorTextField()
void	getFields(RowMetaInterface inputRowMeta, String name, RowMetaInterface[] info, StepMeta nextStep, VariableSpace space) Deprecated.
void	getFields(RowMetaInterface row, String name, RowMetaInterface[] info, StepMeta nextStep, VariableSpace space, Repository repository, org.pentaho.metastore.api.IMetaStore metaStore) Gets the fields.
int	getFileFormatTypeNr()
String[]	getFileName()
String[]	getFilePaths(VariableSpace space)
int	getFileTypeNr()
TextFileFilter[]	getFilter()
String[]	getInfoSteps()
String	getLength()
String	getLookupStepname()
String	getRequiredFilesDesc(String tt)
StepInterface	getStep(StepMeta stepMeta, StepDataInterface stepDataInterface, int cnr, TransMeta transMeta, Trans trans) Get the executing step, needed by Trans to launch a step.
StepDataInterface	getStepData() Get a new instance of the appropriate data class.
FileInputList	getTextFileList(VariableSpace space)
String	getXML() Produces the XML string that describes this step's information.
boolean	isAcceptingFilenames() Required for the Data Lineage.
boolean	isErrorLineSkipped()
protected String	loadSource(Node filenode, Node filenamenode, int i, org.pentaho.metastore.api.IMetaStore metaStore)
protected String	loadSourceRep(Repository rep, ObjectId id_step, int i, org.pentaho.metastore.api.IMetaStore metaStore)
void	loadXML(Node stepnode, List<DatabaseMeta> databases, org.pentaho.metastore.api.IMetaStore metaStore) Load the values for this step from an XML Node
void	readRep(Repository rep, org.pentaho.metastore.api.IMetaStore metaStore, ObjectId id_step, List<DatabaseMeta> databases) Read the steps information from a Kettle repository
void	resolve() Resolves resource data elements variables and temporary references.
void	saveRep(Repository rep, org.pentaho.metastore.api.IMetaStore metaStore, ObjectId id_transformation, ObjectId id_step) Save the steps data into a Kettle repository
protected void	saveSource(StringBuilder retVal, String source)
protected void	saveSourceRep(Repository rep, ObjectId id_transformation, ObjectId id_step, int i, String fileName)
void	searchInfoAndTargetSteps(List<StepMeta> steps) Change step names into step objects to allow them to be name-changed etc.
void	setAcceptingStep(StepMeta acceptingStep)
void	setDefault() Set default values
void	setErrorCountField(String errorCountField)
void	setErrorFieldsField(String errorFieldsField)
void	setErrorLineSkipped(boolean errorLineSkipped)
void	setErrorTextField(String errorTextField)
void	setFileName(String[] fileName)
void	setFileNameForTest(String[] fileName)
void	setFilter(TextFileFilter[] filter)
void	setLength(String length)
boolean	supportsErrorHandling()

Methods inherited from class org.pentaho.di.trans.steps.file.BaseFileInputMeta

getFileInputList, getFilePaths, getInputFields, getRequiredFilesCode, getResourceDependencies, inputFiles_fileRequired, inputFiles_includeSubFolders, inputFiles_includeSubFolders

Methods inherited from class org.pentaho.di.trans.steps.file.BaseFileMeta

passDataToServletOutput, writesToFile

Methods inherited from class org.pentaho.di.trans.step.BaseStepMeta

analyseImpact, analyseImpact, cancelQueries, check, check, createEntry, excludeFromCopyDistributeVerification, excludeFromRowLayoutVerification, exportResources, extractStepMetadataEntries, findAttribute, findParent, findParentEntry, getActiveReferencedObjectDescription, getDescription, getDialogClassName, getLog, getLogChannelId, getName, getObjectCopy, getObjectId, getObjectRevision, getObjectType, getOptionalStreams, getParent, getParentStepMeta, getReferencedObjectDescriptions, getRepCode, getRepositoryDirectory, getRequiredFields, getRequiredFields, getSQLStatements, getSQLStatements, getStepInjectionMetadataEntries, getStepIOMeta, getStepMetaInjectionInterface, getSupportedTransformationTypes, getTableFields, getTooltip, getUsedArguments, getUsedDatabaseConnections, getUsedLibraries, getXmlCode, handleStreamSelection, hasChanged, hasRepositoryReferences, isBasic, isDebug, isDetailed, isReferencedObjectEnabled, isRowLevel, loadReferencedObject, loadReferencedObject, loadStepAttributes, loadXML, loadXML, logBasic, logBasic, logDebug, logDebug, logDetailed, logDetailed, logError, logError, logError, logMinimal, logMinimal, logRowlevel, logRowlevel, lookupRepositoryReferences, readRep, readRep, resetStepIoMeta, saveRep, setChanged, setChanged, setParentStepMeta

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.pentaho.di.trans.step.StepMetaInterface

analyseImpact, analyseImpact, cancelQueries, check, cleanAfterHopFromRemove, excludeFromCopyDistributeVerification, excludeFromRowLayoutVerification, exportResources, extractStepMetadataEntries, getActiveReferencedObjectDescription, getDialogClassName, getOptionalStreams, getParentStepMeta, getReferencedObjectDescriptions, getRequiredFields, getResourceDependencies, getSQLStatements, getSQLStatements, getStepIOMeta, getStepMetaInjectionInterface, getSupportedTransformationTypes, getTableFields, getUsedArguments, getUsedDatabaseConnections, getUsedLibraries, handleStreamSelection, hasChanged, hasRepositoryReferences, isReferencedObjectEnabled, loadReferencedObject, loadXML, lookupRepositoryReferences, passDataToServletOutput, readRep, resetStepIoMeta, saveRep, setChanged, setParentStepMeta

Field Detail

FILE_FORMAT_DOS

public static final int FILE_FORMAT_DOS

See Also:

Constant Field Values

FILE_FORMAT_UNIX

public static final int FILE_FORMAT_UNIX

See Also:

Constant Field Values

FILE_FORMAT_MIXED

public static final int FILE_FORMAT_MIXED

See Also:

Constant Field Values

FILE_TYPE_CSV

public static final int FILE_TYPE_CSV

See Also:

Constant Field Values

FILE_TYPE_FIXED

public static final int FILE_TYPE_FIXED

See Also:

Constant Field Values

content

public TextFileInputMeta.Content content

errorCountField

public String errorCountField

The name of the field that will contain the number of errors in the row

errorFieldsField

public String errorFieldsField

The name of the field that will contain the names of the fields that generated errors, separated by ,

errorTextField

public String errorTextField

The name of the field that will contain the error texts, separated by CR

errorLineSkipped

public boolean errorLineSkipped

If error line are skipped, you can replay without introducing doubles.

Constructor Detail

TextFileInputMeta

public TextFileInputMeta()

Method Detail

getFileName

public String[] getFileName()

Returns:

Returns the fileName.

setFileName

public void setFileName(String[] fileName)

Parameters:

fileName - The fileName to set.

getFilter

public TextFileFilter[] getFilter()

Returns:

The array of filters for the metadata of this text file input step.

setFilter

public void setFilter(TextFileFilter[] filter)

Parameters:

filter - The array of filters to use

loadXML

public void loadXML(Node stepnode,
                    List<DatabaseMeta> databases,
                    org.pentaho.metastore.api.IMetaStore metaStore)
             throws KettleXMLException

Description copied from interface: StepMetaInterface

Load the values for this step from an XML Node

Specified by:

loadXML in interface StepMetaInterface

Overrides:

loadXML in class BaseStepMeta

Parameters:

stepnode - the Node to get the info from

databases - The available list of databases to reference to

metaStore - the metastore to optionally load external reference metadata from

Throws:

KettleXMLException - When an unexpected XML error occurred. (malformed etc.)

clone

public Object clone()

Description copied from interface: StepMetaInterface

Make an exact copy of this step, make sure to explicitly copy Collections etc.

Specified by:

clone in interface StepMetaInterface

Overrides:

clone in class BaseFileInputMeta<BaseFileInputAdditionalField,BaseFileInputFiles,BaseFileField>

Returns:

an exact copy of this step

allocate

public void allocate(int nrfiles,
                     int nrfields,
                     int nrfilters)

allocateFiles

public void allocateFiles(int nrFiles)

setDefault

public void setDefault()

Description copied from interface: StepMetaInterface

Set default values

Specified by:

setDefault in interface StepMetaInterface

getFields

public void getFields(RowMetaInterface row,
                      String name,
                      RowMetaInterface[] info,
                      StepMeta nextStep,
                      VariableSpace space,
                      Repository repository,
                      org.pentaho.metastore.api.IMetaStore metaStore)
               throws KettleStepException

Description copied from class: BaseStepMeta

Gets the fields.

Specified by:

getFields in interface StepMetaInterface

Overrides:

getFields in class BaseStepMeta

Parameters:

row - the input row meta that is modified in this method to reflect the output row metadata of the step

name - Name of the step to use as input for the origin field in the values

info - Fields used as extra lookup information

nextStep - the next step that is targeted

space - the space The variable space to use to replace variables

repository - the repository to use to load Kettle metadata objects impacting the output fields

metaStore - the MetaStore to use to load additional external data or metadata impacting the output fields

Throws:

KettleStepException - the kettle step exception

getFields

@Deprecated
public void getFields(RowMetaInterface inputRowMeta,
                                  String name,
                                  RowMetaInterface[] info,
                                  StepMeta nextStep,
                                  VariableSpace space)
                           throws KettleStepException

Deprecated.

Description copied from class: BaseStepMeta

Gets the fields.

Specified by:

getFields in interface StepMetaInterface

Overrides:

getFields in class BaseStepMeta

Parameters:

inputRowMeta - the input row meta that is modified in this method to reflect the output row metadata of the step

name - Name of the step to use as input for the origin field in the values

info - Fields used as extra lookup information

nextStep - the next step that is targeted

space - the space The variable space to use to replace variables

Throws:

KettleStepException - the kettle step exception

getXML

public String getXML()

Description copied from class: BaseStepMeta

Produces the XML string that describes this step's information.

Specified by:

getXML in interface StepMetaInterface

Overrides:

getXML in class BaseStepMeta

Returns:

String containing the XML describing this step.

getLookupStepname

public String getLookupStepname()

searchInfoAndTargetSteps

public void searchInfoAndTargetSteps(List<StepMeta> steps)

Description copied from class: BaseStepMeta

Change step names into step objects to allow them to be name-changed etc.

Specified by:

searchInfoAndTargetSteps in interface StepMetaInterface

Overrides:

searchInfoAndTargetSteps in class BaseStepMeta

Parameters:

steps - optionally search the info step in a list of steps

getInfoSteps

public String[] getInfoSteps()

readRep

public void readRep(Repository rep,
                    org.pentaho.metastore.api.IMetaStore metaStore,
                    ObjectId id_step,
                    List<DatabaseMeta> databases)
             throws KettleException

Description copied from interface: StepMetaInterface

Read the steps information from a Kettle repository

Specified by:

readRep in interface StepMetaInterface

Overrides:

readRep in class BaseStepMeta

Parameters:

rep - The repository to read from

metaStore - The MetaStore to read external information from

id_step - The step ID

databases - The databases to reference

Throws:

KettleException - When an unexpected error occurred (database, network, etc)

saveRep

public void saveRep(Repository rep,
                    org.pentaho.metastore.api.IMetaStore metaStore,
                    ObjectId id_transformation,
                    ObjectId id_step)
             throws KettleException

Description copied from interface: StepMetaInterface

Save the steps data into a Kettle repository

Specified by:

saveRep in interface StepMetaInterface

Overrides:

saveRep in class BaseStepMeta

Parameters:

rep - The Kettle repository to save to

metaStore - the metaStore to optionally write to

id_transformation - The transformation ID

id_step - The step ID

Throws:

KettleException - When an unexpected error occurred (database, network, etc)

check

public void check(List<CheckResultInterface> remarks,
                  TransMeta transMeta,
                  StepMeta stepMeta,
                  RowMetaInterface prev,
                  String[] input,
                  String[] output,
                  RowMetaInterface info,
                  VariableSpace space,
                  Repository repository,
                  org.pentaho.metastore.api.IMetaStore metaStore)

Description copied from interface: StepMetaInterface

Checks the settings of this step and puts the findings in a remarks List.

Specified by:

check in interface StepMetaInterface

Overrides:

check in class BaseStepMeta

Parameters:

remarks - The list to put the remarks in @see org.pentaho.di.core.CheckResult

stepMeta - The stepMeta to help checking

prev - The fields coming from the previous step

input - The input step names

output - The output step names

info - The fields that are used as information by the step

space - the variable space to resolve variable expressions with

repository - the repository to use to load Kettle metadata objects impacting the output fields

metaStore - the MetaStore to use to load additional external data or metadata impacting the output fields

getStep

public StepInterface getStep(StepMeta stepMeta,
                             StepDataInterface stepDataInterface,
                             int cnr,
                             TransMeta transMeta,
                             Trans trans)

Description copied from interface: StepMetaInterface

Get the executing step, needed by Trans to launch a step.

Specified by:

getStep in interface StepMetaInterface

Parameters:

stepMeta - The step info

stepDataInterface - the step data interface linked to this step. Here the step can store temporary data, database connections, etc.

cnr - The copy nr to get

transMeta - The transformation info

trans - The launching transformation

getStepData

public StepDataInterface getStepData()

Description copied from interface: StepMetaInterface

Get a new instance of the appropriate data class. This data class implements the StepDataInterface. It basically contains the persisting data that needs to live on, even if a worker thread is terminated.

Specified by:

getStepData in interface StepMetaInterface

Returns:

The appropriate StepDataInterface class.

getErrorCountField

public String getErrorCountField()

setErrorCountField

public void setErrorCountField(String errorCountField)

getErrorFieldsField

public String getErrorFieldsField()

setErrorFieldsField

public void setErrorFieldsField(String errorFieldsField)

getErrorTextField

public String getErrorTextField()

setErrorTextField

public void setErrorTextField(String errorTextField)

getRequiredFilesDesc

public String getRequiredFilesDesc(String tt)

isErrorLineSkipped

public boolean isErrorLineSkipped()

setErrorLineSkipped

public void setErrorLineSkipped(boolean errorLineSkipped)

setAcceptingStep

public void setAcceptingStep(StepMeta acceptingStep)

Parameters:

acceptingStep - The acceptingStep to set.

getFileFormatTypeNr

public int getFileFormatTypeNr()

getFileTypeNr

public int getFileTypeNr()

exportResources

public String exportResources(VariableSpace space,
                              Map<String,ResourceDefinition> definitions,
                              ResourceNamingInterface resourceNamingInterface,
                              Repository repository,
                              org.pentaho.metastore.api.IMetaStore metaStore)
                       throws KettleException

Since the exported transformation that runs this will reside in a ZIP file, we can't reference files relatively. So what this does is turn the name of files into absolute paths OR it simply includes the resource in the ZIP file. For now, we'll simply turn it into an absolute path and pray that the file is on a shared drive or something like that.

Specified by:

exportResources in interface StepMetaInterface

Overrides:

exportResources in class BaseStepMeta

Parameters:

space - the variable space to use

definitions -

resourceNamingInterface -

repository - The repository to optionally load other resources from (to be converted to XML)

metaStore - the metaStore in which non-kettle metadata could reside.

Returns:

the filename of the exported resource

Throws:

KettleException

supportsErrorHandling

public boolean supportsErrorHandling()

Specified by:

supportsErrorHandling in interface StepMetaInterface

Overrides:

supportsErrorHandling in class BaseStepMeta

Returns:

true if this step supports error "reporting" on rows: the ability to send rows to a certain target step.

setFileNameForTest

public void setFileNameForTest(String[] fileName)

loadSource

protected String loadSource(Node filenode,
                            Node filenamenode,
                            int i,
                            org.pentaho.metastore.api.IMetaStore metaStore)

saveSource

protected void saveSource(StringBuilder retVal,
                          String source)

loadSourceRep

protected String loadSourceRep(Repository rep,
                               ObjectId id_step,
                               int i,
                               org.pentaho.metastore.api.IMetaStore metaStore)
                        throws KettleException

Throws:

KettleException

saveSourceRep

protected void saveSourceRep(Repository rep,
                             ObjectId id_transformation,
                             ObjectId id_step,
                             int i,
                             String fileName)
                      throws KettleException

Throws:

KettleException

getEncoding

public String getEncoding()

Specified by:

getEncoding in class BaseFileInputMeta<BaseFileInputAdditionalField,BaseFileInputFiles,BaseFileField>

getLength

public String getLength()

Returns:

the length

setLength

public void setLength(String length)

Parameters:

length - the length to set

isAcceptingFilenames

public boolean isAcceptingFilenames()

Required for the Data Lineage.

Overrides:

isAcceptingFilenames in class BaseFileInputMeta<BaseFileInputAdditionalField,BaseFileInputFiles,BaseFileField>

getAcceptingStepName

public String getAcceptingStepName()

Required for the Data Lineage.

Overrides:

getAcceptingStepName in class BaseFileInputMeta<BaseFileInputAdditionalField,BaseFileInputFiles,BaseFileField>

getAcceptingStep

public StepMeta getAcceptingStep()

Required for the Data Lineage.

getAcceptingField

public String getAcceptingField()

Required for the Data Lineage.

Overrides:

getAcceptingField in class BaseFileInputMeta<BaseFileInputAdditionalField,BaseFileInputFiles,BaseFileField>

getFilePaths

public String[] getFilePaths(VariableSpace space)

getTextFileList

public FileInputList getTextFileList(VariableSpace space)

resolve

public void resolve()

Description copied from interface: ResolvableResource

Resolves resource data elements variables and temporary references.

Specified by:

resolve in interface ResolvableResource