public abstract class DBLoader extends Object
Within this class is the Table sub-class. There is an abstract method the returns an array of Table instances. A Table instance is used to define a database table and its content (its rows or data). A concrete subclass of this class can define how the array of Tables are created (CSV files, XML files, by reading another database, etc.) and then using this class load thoses Tables into a database. In addition, rather than loading a database, this class can be used to generate files containing the sql that can be used to load the database.
To use this class the following must be specified:
Each Table object created has a Controller object with four boolean instance variables that control what actions are taken when the Table's executeStatements method is called. Those instance variables are:
The Table.Controller must also have its RowStream object defined. A RowStream produces a set of Row objects (see the Row interface below) which in turn has an array of Objects that represent the values of a row in the Table. The default RowStream is an emtpy RowStrean, no rows. The user must implement the RowStrean interface. One such implementation might be a RowStrean containing a list of rows. In this case, all of the rows would be in memory. Another implementation might read each row's data from a file or another database. In this case, the row data is not in memory allowing one to load much larger tables.
Each column must have one of the following SQL data type definitions:
NOTE: Much of the code appearing in this class came from the MondrianFoodMartLoader class.
Modifier and Type | Class and Description |
---|---|
static class |
DBLoader.Column |
static interface |
DBLoader.Row |
static class |
DBLoader.RowDefault |
static interface |
DBLoader.RowStream
The RowStream interface allows one to load large sets of
rows by streaming them in, one does not have to have all
of the row data in memory.
|
class |
DBLoader.Table |
static class |
DBLoader.Type
Represents a logical type, such as "BOOLEAN".
Specific databases will represent this their own particular physical
type, for example "TINYINT(1)", "BOOLEAN" or "BIT";
see
DBLoader.Type.toPhysical(mondrian.spi.Dialect) . |
Modifier | Constructor and Description |
---|---|
protected |
DBLoader() |
Modifier and Type | Method and Description |
---|---|
protected void |
check() |
void |
clear()
Undoes all of the database table creations performed
when the load method was called.
|
void |
close()
Releases resources.
|
protected void |
closeFileWriter() |
protected String |
columnValue(DBLoader.Column column,
Object value)
Convert the columns value to a string based upon its column type.
|
protected String |
createInsertStatement(DBLoader.Table table,
Object[] values) |
static String |
decimalFormat(int length,
int places)
Generate an appropriate number format string for doubles etc
to be used to include a number in an SQL insert statement.
|
static String |
decimalFormat(String lengthStr,
String placesStr)
Generate an appropriate number format string for doubles etc
to be used to include a number in an SQL insert statement.
|
void |
dropTable(DBLoader.Table table) |
void |
dropTables(DBLoader.Table[] tables) |
protected String |
embedQuotes(String original)
Generate an appropriate string to use in an SQL insert statement for a
VARCHAR colummn, taking into account NULL strings and strings with
embedded quotes
|
protected void |
executeAfterActions(DBLoader.Table table) |
protected void |
executeBeforeActions(DBLoader.Table table) |
protected boolean |
executeCreateTable(DBLoader.Table table) |
protected void |
executeDDL(String ddl) |
protected boolean |
executeDropTable(DBLoader.Table table) |
protected boolean |
executeDropTableRows(DBLoader.Table table) |
protected int |
executeLoadTableRows(DBLoader.Table table) |
protected void |
executeStatements(DBLoader.Table table) |
void |
executeStatements(DBLoader.Table[] tables) |
protected void |
generateAfterActions(DBLoader.Table table) |
protected void |
generateBeforeActions(DBLoader.Table table) |
protected void |
generateCreateTable(DBLoader.Table table) |
protected void |
generateDropTable(DBLoader.Table table) |
protected void |
generateDropTableRows(DBLoader.Table table) |
protected void |
generateStatements(DBLoader.Table table) |
void |
generateStatements(DBLoader.Table[] tables) |
int |
getBatchSize() |
Connection |
getConnection() |
boolean |
getForce() |
String |
getJdbcDriver() |
String |
getJdbcURL() |
File |
getOutputDirectory() |
String |
getPassword() |
abstract DBLoader.Table[] |
getTables() |
String |
getUserName() |
void |
initialize() |
protected void |
initializeColumns(DBLoader.Column[] columns) |
protected boolean |
makeFileWriter(DBLoader.Table table,
String suffix) |
protected String |
quoteId(String name)
Quote the given SQL identifier suitable for the output DBMS.
|
void |
setBatchSize(int batchSize) |
void |
setConnection(Connection connection) |
void |
setForce(boolean force) |
void |
setJdbcDriver(String jdbcDriver) |
void |
setJdbcURL(String jdbcURL) |
void |
setOutputDirectory(File outputDirectory) |
void |
setPassword(String password) |
void |
setUserName(String userName) |
protected int |
writeBatch(String[] batch,
int batchSize)
If we are outputting to JDBC,
Execute the given set of SQL statements
Otherwise,
output the statements to a file.
|
protected void |
writeDDL(String ddl) |
protected static final org.apache.log4j.Logger LOGGER
public static final String nl
public static final String BATCH_SIZE_PROP
public static final String JDBC_DRIVER_PROP
public static final String JDBC_URL_PROP
public static final String JDBC_USER_PROP
public static final String JDBC_PASSWORD_PROP
public static final String OUTPUT_DIRECTORY_PROP
public static final String FORCE_PROP
public static final String DROP_TABLE_INDEX_PROP
public static final String DROP_TABLE_INDEX_SUFFIX_DEFAULT
public static final String CREATE_TABLE_INDEX_PROP
public static final String CREATE_TABLE_INDEX_SUFFIX_DEFAULT
public static final String DROP_TABLE_PROP
public static final String DROP_TABLE_SUFFIX_DEFAULT
public static final String DROP_TABLE_ROWS_PROP
public static final String DROP_TABLE_ROWS_SUFFIX_DEFAULT
public static final String CREATE_TABLE_PROP
public static final String CREATE_TABLE_SUFFIX_DEFAULT
public static final String LOAD_TABLE_ROWS_PROP
public static final String LOAD_TABLE_ROWS_SUFFIX_DEFAULT
static final Pattern decimalDataTypeRegex
static final Pattern varcharDataTypeRegex
static final DecimalFormat integerFormatter
static final String dateFormatString
static final String oracleDateFormatString
static final DateFormat dateFormatter
static final DBLoader.RowStream EMPTY_ROW_STREAM
public static String decimalFormat(String lengthStr, String placesStr)
lengthStr
- String representing integer: number of digits to formatplacesStr
- String representing integer: number of decimal placespublic static String decimalFormat(int length, int places)
length
- int: number of digits to formatplaces
- int: number of decimal placespublic abstract DBLoader.Table[] getTables() throws Exception
Exception
public void dropTables(DBLoader.Table[] tables) throws Exception
Exception
public void dropTable(DBLoader.Table table) throws Exception
Exception
public void setOutputDirectory(File outputDirectory)
public File getOutputDirectory()
public void setForce(boolean force)
public boolean getForce()
public void setBatchSize(int batchSize)
public int getBatchSize()
public void setJdbcDriver(String jdbcDriver)
public String getJdbcDriver()
public void setJdbcURL(String jdbcURL)
public String getJdbcURL()
public void setUserName(String userName)
public String getUserName()
public void setPassword(String password)
public String getPassword()
public Connection getConnection()
public void setConnection(Connection connection)
public void generateStatements(DBLoader.Table[] tables) throws Exception
Exception
protected void generateStatements(DBLoader.Table table) throws Exception
Exception
protected void generateBeforeActions(DBLoader.Table table)
protected void generateDropTable(DBLoader.Table table)
protected void generateDropTableRows(DBLoader.Table table)
protected void generateCreateTable(DBLoader.Table table)
protected void generateAfterActions(DBLoader.Table table)
public void executeStatements(DBLoader.Table[] tables) throws Exception
Exception
protected void executeStatements(DBLoader.Table table) throws Exception
Exception
protected boolean makeFileWriter(DBLoader.Table table, String suffix) throws Exception
Exception
protected void closeFileWriter()
public void clear()
public void close()
Call this method when the load process is finished and the connection is no longer going to be used.
protected void initializeColumns(DBLoader.Column[] columns)
protected void executeBeforeActions(DBLoader.Table table) throws Exception
Exception
protected void executeAfterActions(DBLoader.Table table) throws Exception
Exception
protected boolean executeDropTableRows(DBLoader.Table table) throws Exception
Exception
protected boolean executeDropTable(DBLoader.Table table)
protected boolean executeCreateTable(DBLoader.Table table)
protected int executeLoadTableRows(DBLoader.Table table)
protected String createInsertStatement(DBLoader.Table table, Object[] values) throws Exception
Exception
protected String quoteId(String name)
name
- Identifierprotected String columnValue(DBLoader.Column column, Object value)
column
- Columnvalue
- Column valueprotected String embedQuotes(String original)
original
- String to transformprotected int writeBatch(String[] batch, int batchSize) throws IOException, SQLException
batch
- SQL statements to executebatchSize
- # SQL statements to executeIOException
SQLException