Talend Cloud product architecture
Talend Cloud Services are online services providing web-based interfaces through which you can access features and functions of the Talend Software to design, manage, and monitor integration capabilities.
Information noteNote: Some of the services described below may not be available to you depending on your license or deployment.
The diagram is divided into two main parts: the customer-controlled environment and the Talend Cloud environment.
Customer-controlled environment
The customer-controlled environment includes a web browser, Talend Studio, Talend Cloud API Tester, a Remote Engine or a Remote Engine Gen2, and a Runtime Server.
- From your web browser, you can access the different cloud applications.
- From Talend Studio, you can:
- Publish data integration Jobs to Talend Management Console as tasks, make them available to web users, and run them in the cloud.
- Benefit from the Talend Cloud Data Preparation features through the use of the Data Preparation components. You can create datasets from various databases and export them in Talend Cloud Data Preparation, or leverage a preparation directly in a data integration Job or Spark Job.
- Use Jobs with the Data Stewardship components to load, retrieve, or delete tasks from the campaigns created in the Talend Cloud Data Stewardship.
- Import an API definition from Talend Cloud API Designer to implement it and test it with Talend Cloud API Tester.
- The Remote Engine, the Remote Engine Gen2 and Talend Runtime are used to run artifacts, tasks, preparations, and pipelines on premises.
- In Talend Cloud API Tester, you can create or import requests and scenarios to test your API. You can also automate your tests using the Maven plugin.
Talend Cloud environment
The Talend Cloud environment
includes the cloud applications and the Cloud Engines.
- In Talend Management Console, you can administrate roles, users, projects, and licenses. You can create new users for the cloud applications and assign them to custom groups. You can then define roles and assign them to your users. Talend Management Console is also used to import your license files and create projects to collaborate on in Talend Studio. In addition, you can enable data and file transfer, data integration, and access to shared data sources for web users. For example, you can import and use preconfigured sample tasks, or design tasks that automate the exchange and synchronization of data between applications.
- In Talend Cloud Pipeline Designer, you can design complex end-to-end pipelines to process, enrich, and transform data at rest or in motion. You can run these Pipelines either on premises using a local Remote Engine Gen2 or on the cloud using either the embedded Cloud Engine or a Remote Engine Gen2 installed on your virtual private cloud.
- With Talend Cloud Data Inventory, you can manage and maintain an inventory of data assets managed within Talend Cloud. Datasets are added to this inventory whenever they are collected or created using Talend Cloud Pipeline Designer or Talend Cloud Data Preparation. These datasets are automatically profiled and associated with a Trust Score. They can then be shared so that other users in the Talend Cloud platform can easily find them.
- In Talend Cloud Data Preparation, you can import your data from local files or other sources, and cleanse or enrich it by creating new preparations.
- In Talend Cloud Data Stewardship, campaign owners manage data assets and organize the data interactions whenever human intervention is required to collaborate on data curation, arbitration, or validation.
- In Talend Dictionary Service, you can add, remove, or modify the semantic categories that are applied to each column in your data when opened in Talend Cloud Data Preparation or Talend Cloud Data Stewardship.
- In Talend Cloud API Designer, you can design APIs or import existing definitions, publish their documentation and create mocks. You can then export your definitions in OpenAPI Specification/Swagger or RAML. You can also test your APIs onTalend Cloud API Tester at any step in the design.
The Cloud Engines are used to run artifacts, tasks, preparations, and pipelines in the cloud.