Adding a JDBC driver to your engine

You have the possibility to add a JDBC driver to the Remote Engine Gen2 in order to connect to databases that are not explicitly supported in Talend Cloud Data Inventory.

The generated SQL statements will be SQL-92 compatible.

Procedure

Go to the Remote Engine Gen2 installation directory.
Edit the following file that is the Docker configuration file used when starting the engine:
default/docker-compose.yml if you are using the engine in the AWS USA, AWS Europe, AWS Asia-Pacific or Azure regions.
eap/docker-compose.yml if you are using the engine as part of the Early Adopter Program.

Find the livy container and add this JDBC driver configuration under the environment section:

livy:
    image: ${LIVY_IMG}
    networks:
      default:
        aliases:
          - livy
    environment:
      LIVY_JOB_SPARK_SUBMIT_OPTS: "-Djdbc.drivers[10].id=<DATABASE_ID> -Djdbc.drivers[10].displayName=<DISPLAY_NAME> -Djdbc.drivers[10].order=<ORDER_IN_THE_LIST> -Djdbc.drivers[10].className=<JDBC_DRIVER_CLASS> -Djdbc.drivers[10].paths[0]=<MAVEN GROUP>:<MAVEN ARTIFACT>:jar:<MAVEN VERSION>"
      [...]

where <DATABASE_ID> (the database identifier), <DISPLAY_NAME> (the database display name), <ORDER_IN_THE_LIST> (the position of the database in the list of connections in the app) , <JDBC_DRIVER_CLASS> (the database JDBC driver class) and <MAVEN GROUP>:<MAVEN ARTIFACT>:jar:<MAVEN VERSION> (the database artifact) need to be replaced with your own database information.

Example

For a DB2 database

livy:
    image: ${LIVY_IMG}
    networks:
      default:
        aliases:
          - livy
    environment:
		LIVY_JOB_SPARK_SUBMIT_OPTS: "-Djdbc.drivers[10].id=DB2 -Djdbc.drivers[10].displayName=DB2 -Djdbc.drivers[10].order=900 -Djdbc.drivers[10].className=com.ibm.db2.jcc.DB2Driver -Djdbc.drivers[10].paths[0]=com.ibm.db2:jcc:jar:11.5.0.0"
		[...]

Find the component-server container and add this configuration under the environment/_JAVA_OPTIONS section to add the JDBC driver to the list of connectors and to be able to call the corresponding services:

component-server:
    image: ${COMPONENTS_IMG}
    networks:
      default:
        aliases:
          - component-server
    environment:
      [...]
      _JAVA_OPTIONS: -Xmx${COMPONENTS_MEM} -Dhttp=443
		[...@=]
		-Djdbc.drivers[10].id=<DATABASE_ID>
		-Djdbc.drivers[10].displayName=<DISPLAY_NAME>
		-Djdbc.drivers[10].order=<ORDER_IN_THE_LIST>
		-Djdbc.drivers[10].className=<JDBC_DRIVER_CLASS>
		-Djdbc.drivers[10].paths[0]=<MAVEN GROUP>:<MAVEN ARTIFACT>:jar:<MAVEN VERSION>
		[...]

Example

For a DB2 database

component-server:
    image: ${COMPONENTS_IMG}
    networks:
      default:
        aliases:
          - component-server
    environment:
      [...]
      _JAVA_OPTIONS: -Xmx${COMPONENTS_MEM} -Dhttp=443
        [...@=]
        -Djdbc.drivers[10].id=DB2
        -Djdbc.drivers[10].displayName=DB2
        -Djdbc.drivers[10].order=900
        -Djdbc.drivers[10].className=com.ibm.db2.jcc.DB2Driver
        -Djdbc.drivers[10].paths[0]=com.ibm.db2:jcc:jar:11.5.0.0
		[...]

Find the connectors container image and add this configuration under the environment section to make it download the JDBC driver library:

connectors:
    image: talend/connectors:20200528132950116
    networks:
      - default
    environment:
      CONNECTORS_SETUP_OPTS: setup --component-jdbc-auto-download-drivers
        [...]
        --artifact=<MAVEN GROUP>:<MAVEN ARTIFACT>:jar:<MAVEN VERSION>
        [...]

Example

For a DB2 database

connectors:
    image: talend/connectors:20200528132950116
    networks:
      - default
    environment:
      CONNECTORS_SETUP_OPTS: setup --component-jdbc-auto-download-drivers
        [...]
        --artifact=com.ibm.db2:jcc:jar:11.5.0.0
		[...]

Find the previewrunner container and configure it this way in order to make the data sample preview work from the dataset edition form in Talend Cloud Data Inventory:

previewrunner:
    image: ${PREVIEW_RUNNER_IMG}
    user: root
    command: /opt/talend/app/bin/data-streams-previewrunner-ee-svc -Djdbc.drivers[10].id=<DATABASE_ID>
	-Djdbc.drivers[10].displayName=<DISPLAY_NAME> -Djdbc.drivers[10].order=<ORDER_IN_THE_LIST>
	-Djdbc.drivers[10].className=<JDBC_DRIVER_CLASS> -Djdbc.drivers[10].paths[0]=<MAVEN GROUP>:<MAVEN ARTIFACT>:jar:<MAVEN VERSION>
	-Dhttp.port=80 -Dlogger.resource=dss-logback.xml -Dtalend.component.manager.m2.repository=/opt/talend/connectors
	-Dtalend.component.manager.classloader.container.classesAndPackages=org.talend.components.common.

Example

For a DB2 database

previewrunner:
    image: ${PREVIEW_RUNNER_IMG}
    user: root
    command: /opt/talend/app/bin/data-streams-previewrunner-ee-svc -Djdbc.drivers[10].id=DB2
	-Djdbc.drivers[10].displayName=DB2 -Djdbc.drivers[10].order=900
	-Djdbc.drivers[10].className=com.ibm.db2.jcc.DB2Driver -Djdbc.drivers[10].paths[0]=com.ibm.db2:jcc:jar:11.5.0.0
	-Dhttp.port=80 -Dlogger.resource=dss-logback.xml -Dtalend.component.manager.m2.repository=/opt/talend/connectors
	-Dtalend.component.manager.classloader.container.classesAndPackages=org.talend.components.common.

Save your changes and restart your engine.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here