Talend Data Catalog Bridges
New features and improvements
Feature | Description |
---|---|
Any File System: Parquet files / ignore Databricks Delta Lake managed metadata columns |
Implemented ignorance of the next metadata columns for parquet files import: _change_type, _commit_version, _commit_timestamp. |
Microsoft Azure Cosmos DB: Microsoft Babylon export / Support for Microsoft Fabric lineage between Cosmos DB and PowerBI Artifacts |
MicrosoftAzurePowerBI import bridge: implementation of initial support for PowerQuery function DocumentDB.Contents(), for CosmosDB v1 connector. MicrosoftAzureDataCatalog export: initial support for CosmosDB container lineage. |
Qlik Sense: clarification on insecure URL for Maven dependency |
No longer tries on HTTP if HTTPS does not work. |
Microsoft ADLS File System: support for Private Blob Storage |
|
Microsoft SSAS: permissions / request for additional permission info |
Added required permissions to bridge documentation. |
Oracle Database (via JDBC): some OutputTransformationDataSet have numeric column names |
Fix for statement INSERT INTO with clause LOG ERRORS INTO in Oracle expression parser |
Any Qlik import: improper lineage is extracted for the Qlik resource if the tables have the same names in different projects / Update documentation |
Updated FAQ section for all Qlik bridges. |
Any DI/ETL: shared Bridge Library / Switch YamlParameterFile from yaml to json |
Migrated from yaml to json for parameter output. |
Tableau Server (Repository): option for Tableau Prep import with integrated data flow lineage around the Tableau Hyper files |
Added limitation text, updated grammar. Documentation improvement. |
Erwin Data Modeler export: IDERA ER/Studio import / Merged attribute Date Id in the ER/Studio model is imported as two separate attributes Date Id and Date Id/1 |
ER/Studio import bridge: improvement for attribute unification in logical models. |
Denodo Virtual DataPort: the architecture of importing Metadata from Denodo |
Added a new -sources.skip miscellaneous option. |
Snowflake Database (via JDBC): grammatical improvements |
Fixed of grammatical issues. |
Microsoft ADLS File System: file / Import more properties |
|
MicroStrategy: need clarification on syntax error, unexpected EOS_TOKEN |
Added support for parsing mix of SQL and Microstrategy expressions. |
Google Looker Server: UI ID attribute not extracting for dashboard and look |
Looker import bridge: implement support for the UI ID property, for Looks and Dashboards reports. |
Notable fixes
Issue | Description |
---|---|
Snowflake Database (via JDBC): Source System Definition / Incorrect and duplicated at the database and schema level |
Fixed import of schema description for Catalog-based database. |
Any DDL script import/export: Log / Standards / Quotation marks around referenced object name |
Added quotation marks around referenced object name, fixed grammatical issues. |
Any Databricks: Log / Standards / Quotation marks around referenced object name |
Added quotation marks around referenced object name, fixed grammatical issues. |
Any dbt: Log / Standards / Quotation marks around referenced object name |
Added quotation marks around referenced object name, fixed grammatical issues. |
Any Microsoft SQL Server: Log / Standards / Quotation marks around referenced object name |
Added quotation marks around referenced object name, fixed grammatical issues. |
Any IDERA import/export: Log / Standards / Quotation marks around referenced object name |
Added quotation marks around referenced object name, fixed grammatical issues. |
Amazon Web Services (AWS) Glue ETL (via Apache Spark) — Python script / Consistency: TransformationDataAttribute must have a parent |
Fixed the validation error. Improvements for format method of python string. Added support for join method of python string. Added support for set intersection operator and in python interpreter. Added support for functions list, set, map in python interpreter. |
IDERA ER/Studio: import of a dm1 file gets hung with no further progress |
Fix for End of File with the C++ parser. |
Erwin Data Modeler: can't add the same AssociatedPhysicalObject to ModelObject |
Support for statement COMMENT ON COLUMN. Fixed adding the same tablespace to a table. Fixed printing of similar warnings like The datatype for 'ACTIVE_IND' should be string instead of 'NUMBER(1)'. |
MicroStrategy: miscellaneous / -sf and -sp options are not working |
Fixed -sp and -sf parameters for project. |
Any Informatica import/export: Log / Standards / Quotation marks around referenced object name |
Added quotation marks around referenced object name, fixed grammatical issues. |
Microsoft Azure Power BI: HTTP Header Credential Leak |
MicrosoftAzurePowerBI import bridge: disable debug log message related to PowerQuery function Web.Contents(). |
QlikSense Cloud: close session event |
Fixed NPE for importExpression for ReportField. Added debug messages. |
Databricks Delta Lake Hadoop Hive Database (HCatalog and Metastore via JDBC): the new column appeared in the table with the partition column |
Fixed processing of the # Partitioning section depending on Apache Spark version. |
Tableau Server (Repository) and Cloud: browsing Site IDs does not allow select sites |
Fixed regression in Repository Browser Tree. |
SAP Business Warehouse 4 HANA (BW/4HANA) import/Microsoft Purview export: java.lang.NullPointerException |
Fixed NPE for cases when infocube does not have a source. |
MicroStrategy: extraction is failing with MITI.MIRException: Cannot get Table |
Some tables in customer metadata are corrupted. Added code to skip those tables. |
Apache: Log / Standards / Quotation marks around referenced object name |
Added quotation marks around referenced object name, fixed grammatical issues. |
Microsoft Purview export, Microsoft Azure Power BI (import: NPE during Parsing External Entity: Cannot invoke "String.indexOf(String)" because "<local1>" is null |
ApacheAtlasExport — improve process lineage code ApacheAtlasExport: tentative fix for NPE. |
Oracle (via JDBC): missing stored procedures |
Fix for routine argument name COMMENT in Oracle ddl parser. Support for column name TIME in Oracle expression analyzer. Support for MOD operator and MOD function in Oracle expression analyzer. Support for label WHILE in Oracle expression analyzer. |
Amazon Web Services (AWS) Glue ETL (via Apache Spark): no data flow lineage |
Support for method drop_fields of DynamicFrame. Improvements for method writeFrame of DataSink. |
Microsoft Azure Power BI: summarizer bug with Excel files |
Improved support for Excel files/sheets data sources, stitch by name. |
Erwin Data Modeler export, IDERA ER/Studio import: wrong File handle browse:// on export |
Fixed support for browse files in the export bridges. |
Oracle (via JDBC): literal update |
Updated literals with grammatical changes. |
Informatica Cloud Data Quality (ICDQ): MITI.MIRException: The Json schema does not match the expected schema |
Skipped objects with missing mandatory fields in metadata. |
QlikSense Cloud: close session event |
Updated FAQ and Miscellaneous parameter documentation. |
Microsoft SSAS / XMLA / Measure Name displays incorrectly in Calculated Measures. |
-fixed Measure name. |
Microsoft SSAS: column properties Type Length= - 1 not supported |
-fixed -1 type length. |
Snowflake Database (via JDBC) import, Microsoft Purview export: why is the bridge querying SNOWFLAKE.ACCOUNT_USAGE.POLICY_REFERENC ES PR |
Skipped importing the masking policy metadata when -purview option is specified. |
Erwin Data Modeler export, IDERA ER/Studio import: logical only attribute hidden in the source data model was imported as a regular attribute |
CaErwin9Xml export bridge: consider generate property when exporting attributes. |
MicroStrategy: miscellaneous / -sf and -sp options are not working in MSTR default project |
Originally -sp and -sf worked only for standalone filters and prompts and did not work for filters and prompts embedded inside reports. Code has been implemented to skip importing filters and prompts embedded inside reports. |
IBM Cognos: MySQL queries with cast to UTF8 do not show lineage |
CognosReportStudio import bridge — support specifying the connection type as miscellaneous parameter option for example: Miscellaneous: -connection.cast "GRTGCS_MYSQL_DS"="MYSQL". |
Microsoft SQL Server Integration Services (SSIS): documentation / Internal parametrization support |
Added a tooltip for -execution_script miscellaneous option. |
Amazon Web Services (AWS) Athena Database (via JDBC): files being created in S3 locations / Documentation Update |
Updated bridge documentation: Support and FAQ sections have been updated. |
Microsoft Azure Synapse Analytics (SQL Data Warehouse) (via JDBC): no suitable driver found / Improve bridge documentation |
Updated bridge documentation. |
Tableau Server (repository): restoring from the backup doesn't work for one workbook |
Fixed encoding for the files from Tableau Server backups. |