Monitoring data evolution

To keep track of the quality of the address data you initially profiled, you can simply use the Talend Studio to generate a Job which can launch the evolution report you created on the column analysis. You can then deploy this Job on Talend Administration Center and schedule it to run monthly, for example.

This way the report generated from Talend Studio will be launched remotely from Talend Administration Center.

Prerequisites to monitor data evolution

Procedure

Create a data quality project from Talend Administration Center Web Application, dq_proj for example.

For more information about how to create a project in the Talend Administration Center Web Application, see Adding a project.

Information noteNote: If you do not have the required rights to create or manage a project, contact the administrator of your Web Application.
Establish a connection from Talend Studio to the remote SVN repository storing the above-mentioned project.

For more information about how to connect to a remote repository, see Accessing remote projects from Talend Studio.
Launch Talend Studio using this new connection and the new data quality project.

Generating a Job to run the report

In this section you will see how to generate a ready-to-use Job to launch the report you created on the column analysis.

Procedure

In the Profiling perspective of Talend Studio, right-click the profile_customers report under the Report node and select Generate Jobs > Launch a report.

This generates a launching-report Job and opens it in the Integration perspective.

The Report filenames field points to the technical path of the report, and the Output Folder field points to the folder where to save the report file.
Press F6 to execute the Job from Talend Studio for testing purposes.

Results

The Talend Studio console shows information about the report including the report identification, its name and its execution time.

Example of a report from the Talend Studio console.

Creating an execution task and scheduling it

This section describes how to create an execution task in Talend Administration Center in order to execute the launch-report Job created in Talend Studio. It also describes how to define a trigger to launch the execution task once per month.

Procedure

Connect to the Talend Administration Center Web Application.
In the menu tree view of the Web Application, expand Conductor and click Job Conductor to display the execution task list.
From the toolbar on the Job Conductor page, clickAdd to clear the Execution task configuration panel.
On this configuration panel, set the parameters required for executing the launch-report Job as the following:
1. In the Label field, type in the task name.
2. In the Project field, select the data quality project in which the launch-report Job was created.
3. In the Branch field, select trunk as the branch of this project.
4. From the Name list, select the launch-report Job to be used.
5. In the Version list, select the Job version you want to launch; and in the Context field, select the context in which to run the Job.
6. From the Execution server list, select the server which you want to use to execute this task.
Click Save to validate the configuration of this execution task.

The new task is displayed in the Job Conductor page under the data quality project.
From the task list, select the newly added task, click Triggers.
Click Add trigger > Add CRON trigger.
In the Cron Trigger configuration panel, fill in a name for the trigger and click Open UI configurer.
Select the minute, hour, and date at which to execute the task and click Apply modifications.

The selected data is displayed in the trigger configuration panel.

This trigger means that the evolution report will be re-generated at 3:15 PM of the first day of each month.
Click Save.

Deploying the task on the server

Procedure

Select the task in the Job Conductor page and click Generate on the toolbar.
Once the status of this task reads Ready to send on the task list, select the task again and click Deploy on the toolbar.
Once the status of this task reads Ready to run on the task list, select the task again and click Run on the toolbar.

This task will automatically run the launch-report Job from Talend Administration Center Web Application on monthly basis. The column analysis listed in the evolution report is executed and its results are saved in the data quality data mart and the report file is saved in the output folder defined in the tDqReportRun basic settings.

Generating this evolution report repeatedly will track data changes in the address columns initially profiles. The line in the report will start to go upwards if data is fixed and downwards if data gets less accurate and consistent.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here