Package org.pentaho.di.job.entries.ftp

Class MVSFileParser

java.lang.Object

com.enterprisedt.net.ftp.FTPFileParser

org.pentaho.di.job.entries.ftp.MVSFileParser

public class MVSFileParser
extends com.enterprisedt.net.ftp.FTPFileParser

MVS Folder Listing Parser The purpose of this parser is to be able handle responses from an MVS z/OS mainframe FTP server. Many places on the 'net were consulted for input to this parser. Importantly, this information from com.os.os2.networking.tcp-ip group: http://groups.google.com/group/comp.os.os2.networking.tcp-ip/msg/25acc89563f1e93e http://groups.google.com/group/comp. os.os2.networking.tcp-ip/browse_frm/thread/11af1ba1bc6b0edd?hl=en&lr&ie=UTF-8&oe=UTF -8&rnum=6&prev=/groups?q%3DMVS%2BPartitioned %2Bdata%2Bset%2Bdirectory%26hl%3Den%26lr%3D%26ie%3DUTF-8%26oe%3DUTF-8%26selm %3D4e7k0p%2524t1v%2540blackice.winternet.com%26rnum%3D6&pli=1 http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/F1AA2032/1.5.15?SHELF=&DT=20001127174124 Implementation Details 1- This supports folders and partitioned data sets only. This does not support JCL or HFS 2- You must treat partitioned data sets (Dsorg PO/PO-E) like folders and CD to them 3- Dsorg=PS is a downloadable file as are all the contents of a Partitioned Data Set. 4- When downloading from a folder, the Recfm must start with V or F. Note - the location for this is completely up for debate. I modeled this after the ftpsget/FTPSConnection and how ftpsput reaches up and into the ftpsget package to get it. However, I think a better solution is to have an entry/common. James and I agreed (in Matt's absense) to model the behavior after something already existing rather than introduce a new folder (like entry/common or entry/util).

Author:

mbatchelor September 2010

Field Summary

Fields inherited from class com.enterprisedt.net.ftp.FTPFileParser

ignoreDateParseErrors

Constructor Summary

Constructors

Constructor	Description
MVSFileParser(org.pentaho.di.core.logging.LogChannelInterface log)

Method Summary

Modifier and Type	Method	Description
protected boolean	checkDateFormat(String dateStr)
String	getDateFormatString()	Returns the date format string in use for parsing date in the listing.
protected void	guessDateFormat(String dateStr)	This method will look at the incoming date string and try to figure out the format of the date.
boolean	isPartitionedDataset()
protected boolean	isValidDirectoryFormat(String[] listing)	Returns true if this seems to be a recognized MVS folder (not PDS) listing.
boolean	isValidFormat(String[] listing)	Abstract Class Implementations
protected boolean	isValidPDSFormat(String[] listing)	Returns true if this seems to be a recognized MVS PDS listing (not folder).
com.enterprisedt.net.ftp.FTPFile	parse(String raw)	This parses an individual line from the directory listing.
protected com.enterprisedt.net.ftp.FTPFile	parseFolder(String[] aLine, String raw)	Parses a line from a folder listing.
protected com.enterprisedt.net.ftp.FTPFile	parsePDSLine(String[] aLine, String raw)	Parses a Partitioned Dataset Entry, and returns an FTPFile object.
void	setDateFormatString(String value)	Provides ability to pre-specify the format that the parser will use to parse dates.
void	setLocale(Locale arg0)	Could in theory be used to figure out the format of the date/time except that I'd need time on the server to see if this actually works that way.
protected String[]	splitMVSLine(String raw)	This is a split + trim function.
String	toString()	Returns parser name.

Methods inherited from class com.enterprisedt.net.ftp.FTPFileParser

isMultiLine, setIgnoreDateParseErrors, split, split, split, trimStart

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Details

MVSFileParser

public MVSFileParser(org.pentaho.di.core.logging.LogChannelInterface log)

Method Details

isValidFormat

public boolean isValidFormat(String[] listing)

Abstract Class Implementations

Overrides:

isValidFormat in class com.enterprisedt.net.ftp.FTPFileParser

parse

public com.enterprisedt.net.ftp.FTPFile parse(String raw)
                                       throws ParseException

This parses an individual line from the directory listing.

Specified by:

parse in class com.enterprisedt.net.ftp.FTPFileParser

Throws:

ParseException

setLocale

public void setLocale(Locale arg0)

Could in theory be used to figure out the format of the date/time except that I'd need time on the server to see if this actually works that way. For now, we ignore the locale and try to figure out the date format ourselves.

Specified by:

setLocale in class com.enterprisedt.net.ftp.FTPFileParser

toString

public String toString()

Returns parser name. By extensibility oversight in the third-party library we use, this isn't used to match the on the server (unfortunately).

Overrides:

toString in class Object

parsePDSLine

protected com.enterprisedt.net.ftp.FTPFile parsePDSLine(String[] aLine,
 String raw)
                                                 throws ParseException

Parses a Partitioned Dataset Entry, and returns an FTPFile object.

Parameters:

aLine - Split line

raw - Unparsed raw string

Returns:

FTPFile unless it's the header row.

Throws:

ParseException

parseFolder

protected com.enterprisedt.net.ftp.FTPFile parseFolder(String[] aLine,
 String raw)

Parses a line from a folder listing. Note: Returns NULL if it's the header line, if it is ARCIVE or Migrated, if the record format doesn't start with 'F' or 'V', and if the dsorg doesn't start with 'P'.

Parameters:

aLine - Line split apart

raw - Raw line from the transport

Returns:

FTPFile for the line unless it is expressly exluded

splitMVSLine

protected String[] splitMVSLine(String raw)

This is a split + trim function. The String.split method doesn't work well if there are a multiple contiguous white-space characters. StringTokenizer handles this very well. This should never fail to return an array, even if the array is empty. In other words, this should never return null.

Parameters:

raw - The string to tokenize from the MainFrame

Returns:

String array of all the elements from the parse.

isValidDirectoryFormat

protected boolean isValidDirectoryFormat(String[] listing)

Returns true if this seems to be a recognized MVS folder (not PDS) listing.

Parameters:

listing -

Returns:

true if by all appearances this is a listing of an MVS folder

isValidPDSFormat

protected boolean isValidPDSFormat(String[] listing)

Returns true if this seems to be a recognized MVS PDS listing (not folder).

Parameters:

listing -

Returns:

true if by all appearances this is a listing of the contents of a PDS

checkDateFormat

protected boolean checkDateFormat(String dateStr)

guessDateFormat

protected void guessDateFormat(String dateStr)

This method will look at the incoming date string and try to figure out the format of the date. Googling on the internet showed several possible looks to the date: dd/MM/yy yy/MM/dd MM/dd/yy yyyy/MM/dd yyyy/dd/MM I never saw samples showing dd/MM/yyyy but I suppose it's possible. Not happy with this algorithm because it feels clumsy. It works, but it's not very elegant (time crunch).

Parameters:

dateStr -

isPartitionedDataset

public boolean isPartitionedDataset()

Returns:

true if listing is a PDS

getDateFormatString

public String getDateFormatString()

Returns the date format string in use for parsing date in the listing.

Returns:

string format

setDateFormatString

public void setDateFormatString(String value)

Provides ability to pre-specify the format that the parser will use to parse dates.

Parameters:

value - the string to set.