Skip to main content Skip to complementary content

What's new in R2022-04

Big Data: new features

Feature

Description

Available in
Support of Cloudera Data Engineering experience with AWS on Spark Universal 3.1.x You can now run your Spark Jobs on Cloudera Data Engineering experience using Spark Universal with Spark 3.1.x. You can configure it in the Spark Configuration view of your Spark Jobs.

All subscription-based Talend products with Big Data
Support of Databricks runtime 8.x and 9.x with job clusters on Spark Universal 3.1.x You can now run your Spark Jobs on a job Databricks cluster using Spark Universal with Spark 3.1.x. You can configure it either in the Spark Configuration view of your Spark Jobs or in the Hadoop Cluster Connection metadata wizard.

When you select this mode, Talend Studio is compatible from Databricks 8.x to 9.x versions.

All subscription-based Talend products with Big Data
Availability-noteBeta
Support of Spark Universal 3.2.x in Local mode		 You can now run your Spark Jobs using Spark Universal with Spark 3.2.x in Local mode. You can configure it either in the Spark Configuration view of your Spark Jobs or in the Hadoop Cluster Connection metadata wizard.

When you select this mode, Talend Studio builds the Spark environment in itself at runtime to run the Job locally in the Talend Studio.

Talend Data Mapper Big Data components (tHConvertFile, tHMapFile, tHMapInput, and tHMapRecord) are not compatible with Spark 3.2.x for the moment.

As it is a beta feature only, it is not suitable for production environment.

All subscription-based Talend products with Big Data
Support of Google Dataproc 2.0.x on Spark Universal 3.1.x You can now run your Spark Jobs on a Google Dataproc cluster using Spark Universal with Spark 3.1.x. You can configure it in the Spark Configuration view of your Spark Jobs.

When you select this mode, Talend Studio is compatible with Dataproc 2.0.x version on Spark Batch and Spark Streaming Jobs only. Standard Jobs with Hive components are not supported for the moment.

All subscription-based Talend products with Big Data
Support of multiple tGSConfiguration components in Spark Jobs You can now use multiple tGSConfiguration components in one Job to use different Google Storage buckets with different credentials. This feature applies to both Spark Batch and Spark Streaming Jobs.

All subscription-based Talend products with Big Data
Support of CDP Public Cloud with Knox and Impala in Standard Jobs If you use CDP Public Cloud to run your Standard Jobs, you can now make use of both Knox and Impala.

All subscription-based Talend products with Big Data
Support of AWS EMR 6.3.x, 6.4.x and 6.5.x distributions with Spark Universal 3.1.x You can now run your Spark Jobs on an AWS EMR cluster using Spark Universal with Spark 3.1.x in Yarn cluster mode. You can configure it either in the Spark Configuration view of your Spark Jobs or in the Hadoop Cluster Connection metadata wizard.

When you select this mode, Talend Studio is compatible with AWS EMR 6.3.x, 6.4.x and 6.5.x versions. Spark Jobs are not working with tHiveConnection component for the moment.

All subscription-based Talend products with Big Data
Availability-noteBeta
Support of Open Source Apache Spark Standalone cluster on Spark Universal 3.2.x		 You can now run your Spark Jobs in Standalone mode with Spark Universal 3.2.x. You can configure it in the Spark Configuration view of your Spark Jobs.

When you select this mode, Talend Studio connects to a Spark-enabled cluster to run the Job from this cluster.

  • As it is a beta feature only, it is not suitable for production environment.
  • Kerberos is not available yet for beta.

All subscription-based Talend products with Big Data
Availability-noteBeta
Support of Azure Active Directory (AD) Service Principal for Azure Synapse Spark pools in Spark Jobs

When you use Azure Synapse Analytics with Apache Spark pools as a distribution for your Spark Batch and Spark Streaming Jobs, you can now authenticate with Azure AD Service Principal. You can configure it in the Spark Configuration view of your Spark Jobs.

As it is a beta feature only, it is not suitable for production environment.

All subscription-based Talend products with Big Data

Data Integration: new features

Feature

Description

Available in

Talend Studio can now automatically create test branches to test the migration of remote projects

 Talend Studio now provides a new Migrate projects or test wizard with the following two options:
  • Migrate: migrates the project without testing.
  • Test: tests the project migration by creating test branches and migrating the project on test branches to keep the production branch safe.

The wizard is displayed when you open a remote project whose version is lower than the Talend Studio version.

All subscription-based Talend products with Talend Studio

New component to write data to an active ADSO through SAP Java Connector version 3

This release provides the tSAPADSOOutput component, which allows you to write data to an active Advanced Data Store Object (ADSO) through SAP Java Connector version 3.

All subscription-based Talend products with Talend Studio

New components to list and delete files respectively on a Samba host

This release provides the following two components.

  • tSambaList, which lists files in a directory on a Samba v2 or v3 host and passes the information about the files to the subsequent component.

  • tSambaDelete: which deletes a file in a directory on a Samba v2 or v3 host or the entire directory.

All subscription-based Talend products with Talend Studio

Support for Microsoft SQL Server version 2019

This release provides support for Microsoft SQL Server version 2019, which allows Azure Active Directory authentication when connecting a Microsoft SQL server database.

All subscription-based Talend products with Talend Studio

Support for Redshift driver v2.x

This release provides support for Redshift driver v2.x. A Driver version option is added to these Redshift components: tRedshiftInput, tRedshiftUnload, tRedshiftOutput, tRedshiftConnection, tRedshiftRow, tRedshiftOutputBulkExec, tRedshiftBulkExec.

The way you specify JDBC parameters varies between driver v1 and driver v2. For details, see tRedshiftConnection Standard properties.

All subscription-based Talend products with Talend Studio
Support for Dynamic Schema for tSAPTableInput The Dynamic Schema functionality is now supported for tSAPTableInput in your Standard Jobs with the use of tSetDynamicSchema. The Dynamic Schema functionality allows you to configure a schema in a non-static way, so you won't have to redesign your Job for future schema alteration while ensuring it will work all the time. You are now able to provide tables, columns and joins dynamically.

All subscription-based Talend products with Talend Studio

Data Quality: new features

Feature

Description

Available in
New key derivation function The data privacy components have been enhanced with the PBKDF2 key derivation function with 300,000 iterations. This function is the default one and will be used for new Jobs.

For compatibility reasons with Jobs prior to 8.0 R2022-04, PBKDF2 with 65,536 iterations is still supported but deprecated.

Select the key derivation function in the Advanced settings.
The new function is available in:
  • tDataEncrypt
  • tDataDecrypt
  • tDataMasking
  • tDataUnmasking
  • tPatternMasking
  • tPatternUnmasking

All Talend Platform and Data Fabric products

Application Integration: new features

Feature

Description

Available in

The default Microservice monitoring endpoints are now limited to info, health and Jolokia.

Microservices built with Talend Studio provide access to Spring boot actuator endpoints. The default monitoring endpoints are now limited to info, health and Jolokia.

All subscription-based Talend products with ESB

Continuous Integration: new features

Feature

Description

Available in
Talend CI Builder upgraded to version 8.0.4 Talend CI Builder is upgraded from version 8.0.1 to version 8.0.4.

Use Talend CI Builder 8.0.4 in your CI commands or pipeline scripts from this monthly version onwards until a new version of Talend CI Builder is released.

All subscription-based Talend products with Talend Studio
New command to install Talend CommandLine in a standalone CI step A new mvn org.talend.ci:builder-maven-plugin:8.0.4:install command is now available, which allows you to install Talend CommandLine in a standalone CI step without creating a project or generating POM files.

All subscription-based Talend products with Talend Studio

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here