Import from Data Mapping (SQL like) Scripts

Any data mapping with replication mappings and/or query mappings can be imported (without loss) from the Data Mapping Scripts format. This format is based on the standard database SQL Data Manipulation Language (SQL DML) syntax and includes both:

• The data connection data models (e.g. database schema, tables, columns) of their source and target data stores faithfully representing any supported technology (RDBMS, NoSQL, File Systems)
• The data integration (DI/ETL/ELT/CDC) for the data flow lineage between these data stores.

The specifics of the syntax for the data mapping script format are explained in a sample file at:

$MM_HOME/conf/MIRModelBridgeTemplate/DataMappingScript/DataMappingScriptTutorial.sql

The data mapping scripts can be edited /modified or may be generated entirely from scratch to model (simulate) a DI/ETL/ELT/CDC tool which may not be a part of the supported tools for a native model import.

The data mapping scripts can then be

• imported as independent DI models where you may
  • Stitch the model’s connections to any other models (not just the ones originally directly linked to the data mapping
  • Schedule re-harvesting of the scripts to include changes and support automatic stitching associated with configuration management
• imported back into a data mapping
  • where you may continue to benefit from the data mapper UI
  • Import into an existing data mapping, even the one you exported from
  • Import into a newly created data mapping.

Note, the requirements for the second user case (importing back into a data mapping) include a requirement that the connections to the original models must match exactly by pathname. I.e., a data mapping in the repository and its source models and a target model that are a part of the configuration that the data mapping is in. Also, these source and target models are linked by non editable connections, so this connection must be made at the time of the import. I.e., there is a great deal of intelligence built into this import bridge and it hunts for identical structure, but cannot simply “guess” or “make something up” in terms of which source and target models to connect to.

If there is no valid source or target model in the configuration that matches by name (and structure) with what is defined in the data mapping script, then the import log will contain errors explaining this fact. You can, of course, reconnect the end-points to another model with identical structure to the edited structure in the script file.

Finally, generally the best practice, when you are NOT using the data mapping editor to make changes but instead editing the data mapping script source, is to import into a model (not a data mapping model) and thus creating an ETL/DI model that can be stitched and take advantage of support for change and configuration management.