CDC in traditional databases

Talend provides CDC support for all the traditional relational databases. Talend uses a subscriber/publish architecture wherein publisher captures the change data and makes it available to the subscribers. The two main CDC modes are the Trigger mode and Redo/Archive logs mode. Almost all the relational databases use Trigger mode except for Oracle which uses Redo/Archive log mode for capturing changes in the source data.

For Trigger mode to work, the database needs to be already enabled with CDC. This is typically done by stored procedures within that database. For example, in SQL Server, CDC is enabled by executing sys.sp_cdc_enable_db.

Talend needs its own metadata for CDC to work. These are the SUBSCRIBER table, which tracks the tables for changes and the Change table which tracks the changes to the data in the table itself. This is primarily done by creating CDC Metadata in the repository and using the metadata in the Talend CDC components. Talend provides CDC components such as tOracleCDC, tAS400CDC, tDB2CDC, etc.

For more information about Change Data Capture, see Change Data Capture (CDC).

CDC can also be achieved by storing the date and time of when the record was last updated. Then the filter can be applied to the extraction layer of the source system.