List of configuration parameters for Talend Data Preparation
All parameters in the application.properties file are set by default during the installation of Talend Data Preparation by Talend Installer. However, you can customize them according to your installation environment.
For further information about installing and configuring Talend Data Preparation, see Talend installation guides.
Default configuration parameters for the 8.0 version
| Parameter | Description |
|---|---|
| dataprep.locale | Setting for the application interface language. |
| public.ip=<place_your_public_ip_here> server.port=9999 management.server.port=9999 async-runtime.contextPath=/api |
IP of the server hosting Talend Data Preparation and server port. |
| server.compression.enabled=true server.compression.mime-types=text/plain,text/html,text/css,application/json,application/x-javascript,text/xml,application/xml,application/xml+rss,text/javascript,application/javascript,text/x-js |
Response compression parameters. |
| iam.ip=<place_your_iam_ip_here> iam.uri=http://${iam.ip}:9080 iam.api.uri=http://${iam.uri}:9090 |
IP of the server hosting Talend Identity and Access Management, used for SSO, and server port. |
| spring.mvc.async.request-timeout=600000 | Timeout setting for asynchronous executions. Do not change this value, unless you were told so by Talend. |
| dataprep.event.listener=spring spring.service.name=TDP |
Event propagation parameter. Can be spring or Kafka. |
| spring.cloud.function.definition=internalMessage;internalMessageDatasetModule;tsdEventConsumer;tsdSemanticCacheUpdate | Kafka settings. |
| spring.data.mongodb.host=localhost spring.data.mongodb.port=27017 spring.data.mongodb.database=dataprep spring.data.mongodb.username=dataprep-user spring.data.mongodb.password=<randomly_generated_password> multi-tenancy.mongodb.active=false |
MongoDB settings. |
| spring.data.mongodb.uri= | For more complex use cases, mongo.* configurations can be overwritten by specifying a URI directly. |
| mongo.index.tdp_dataSetMetadata.name=dataSetMetadata.name mongo.index.tdp_dataSetMetadata.lifecycle-importing.creation-date=creation-date mongo.index.tdp_acl.resource-id-type=id-type mongo.index.tdp_acl.resource-type.owner-id.controls-type-who=type-who mongo.index.tdp_folder.owner-id.path=id.path mongo.index.tdp_folder.parent-id=parent-id mongo.index.tdp_folderEntry.folder-id.content-type=content-type mongo.index.tdp_folderEntry.content-id.content-type=content-type2 mongo.index.tdp_identifiable.class.content-id=content-id mongo.index.tdp_fullrun.group=group |
Mongo index names. Indicate new index name to get a smaller name than the generated default one. |
| mongodb.ssl=true mongodb.ssl.trust-store=/path/to/trust-store.jks mongodb.ssl.trust-store-password=trust-store-password |
Uncomment these parameters to set up a secured connexion with MongoDB. |
| tls.key-store=/path/to/key-store.jks tls.key-store-password=key-store_password tls.trust-store=/path/to/trust-store.jks tls.trust-store-password=trust-store_password tls.verify-hostname=false |
Uncomment these parameters to set up a secured https connexion for Talend Data Preparation. |
| security.provider=oauth2 security.token.secret=top-secret |
Authentication parameters. |
| talend.security.token.value=<CHANGE_IT> | Token to access actuator data. |
| spring.profiles.active=server-standalone spring.mvc.favicon.enabled=false spring.servlet.multipart.max-file-size=200000000 spring.servlet.multipart.max-request-size=200000000 |
Spring parameters. Do not change these values, unless you were told so by Talend. |
| springdoc.server.url=${gateway-api.service.path} springdoc.swagger-ui.path=/swagger-ui/ springdoc.swagger-ui.disable-swagger-default-url=true springdoc.swagger-ui.docExpansion=none springdoc.swagger-ui.operationsSorter=alpha springdoc.swagger-ui.tagsSorter=alpha springdoc.swagger-ui.urls[0].name=semanticservice springdoc.swagger-ui.urls[0].display-name=Talend Data Preparation - Semantic Service springdoc.swagger-ui.urls[0].url=${gateway-api.service.path}/dq/semanticservice/api-docs spring.mvc.pathmatch.matching-strategy = ant_path_matcher |
Set these parameters to enable access to Swagger. |
| dataset.records.limit=10000 dataset.local.file.size.limit=2000000000 dataset.imports=local,job,tcomp-JDBCDatastore,tcomp-SimpleFileIoDatastore,tcomp-SalesforceDatastore,tcomp-S3Datastore,tcomp-AzureDlsGen2BlobDatastore dataset.list.limit=10 |
Size limit and display parameters for your datasets. |
| api.service.url=http://${public.ip}:${server.port} dataset.service.url=http://${public.ip}:${server.port} tdc.dataset.url=http://${public.ip}:${server.port} dataset-dispatcher.service.url=http://${public.ip}:${server.port} transformation.service.url=http://${public.ip}:${server.port} preparation.service.url=http://${public.ip}:${server.port} fullrun.service.url=http://${public.ip}:${server.port} gateway.service.url=http://${public.ip}:${server.port} tdc.sharing.url=http://${public.ip}:${server.port} tdc.rating.url=http://${public.ip}:${server.port} |
Address of the dataset service. |
| dataset.metadata.store=mongodb preparation.store=mongodb user.data.store=mongodb folder.store=mongodb upgrade.store=mongodb |
File storage service configuration parameters. Do not change these values, unless you were told so by Talend. |
| content-service.store=local content-service.store.local.path=data/ content-service.journalized=true |
Location for cache and content storage. |
| preparation.store.remove.hours=24 | Preparation service configuration. Do not change these values, unless you were told so by Talend. |
| lock.preparation.store=mongodb lock.preparation.delay=600 |
Lock duration parameter in seconds, when working on shared preparations. |
| luceneIndexStrategy=singleton | Lucene index configuration. Do not change these values, unless you were told so by Talend. |
| execution.store=mongodb async.operation.concurrent.run=5 |
Parameters for asynchronous full run and sampling operations, namely storage and number of allowed concurrent runs. Do not change the mongodb value, unless you were told so by Talend. Regarding asynchronous operations, if there are more full run operations than the parameter's value running in parallel, the operations will be queued, and will resume when there is an available slot. You can increase the value of this parameter, according to your machine's power. |
| tcomp.server.url=http://<place_tcomp_ip_here>:8989/tcomp | URL of the server hosting the Components Catalog, used to configure self service connectors. |
| tcomp-SimpleFileIoDatastore.test_connection.visible=false | Parameter to remove the test connection step from the Talend component form. Do not change this parameter, unless you were told so by Talend. |
| async.operation.watcher.ttl=3600000 | Maximum execution time for full runs, in milliseconds. |
| receivers.timeout=3600000 | Maximum waiting time for live datasets input. Information noteWarning: Starting
with the 8.0 version, the live dataset feature is not available
anymore.
|
| dataquality.semantic.list.enable=true dataquality.server.url=<place_data-quality_server_url_here> |
Parameters to activate the semantic type edition in the Talend Data Preparation interface, and URL of the server hosting Talend Dictionary Service. |
| tsd.enabled=true tsd.maven.connector.s3Repository.bucket-url=<place_minio_bucket-url_here> For example: s3://talend-app-shared. tsd.maven.connector.s3Repository.base-path=<place_minio_base-path_here> For example: semantic-types-repository. This property is optional and can be left empty. tsd.maven.connector.s3Repository.username=<place_minio_username_here> tsd.maven.connector.s3Repository.password=<place_minio_password_here> tsd.maven.connector.s3Repository.s3.region=<place_minio_region> For example: us-east-1. tsd.maven.connector.s3Repository.s3.endpoint=<place_minio_server_url_here> For example: https://minio-company.com. tsd.dictionary-provider-facade.producer-url=<place_data-quality_server_url_here> tsd.dictionary-provider.index-folder=tsd-index tsd.maven.connector.temporaryFolder=tsd-index dataquality.event.store=mongodb spring.cloud.stream.kafka.binder.brokers=<place_kafka_ip_here> schema.kafka.topics.prefix= |
Parameters that must match you MinIO or S3 repository configuration in order to use the default and custom semantic types. |
|
spring.security.oauth2.resourceserver.auth0.jwt.issuer-uri=https://talend.auth0.com/ spring.security.oauth2.resourceserver.iam.jwt.jwk-set-uri=${iam.uri}/oidc/jwk/keys spring.security.oauth2.resourceserver.iam.opaque-token.introspection-uri=${iam.uri}/oidc/oauth2/introspect spring.security.oauth2.resourceserver.iam.opaque-token.query-entitlements=true spring.security.oauth2.resourceserver.iam.opaque-token.clientId=<security client id> spring.security.oauth2.resourceserver.iam.opaque-token.clientSecret=<security client secret> spring.security.oauth2.client.registration.oidc.clientId=<security client id> spring.security.oauth2.client.registration.oidc.clientSecret=<security client secret> spring.security.oauth2.client.registration.oidc.authorization-grant-type=authorization_code spring.security.oauth2.client.registration.oidc.scope=openid,refreshToken spring.security.oauth2.client.registration.oidc.redirectUri=http://${public.ip}:${server.port}/signIn spring.security.oauth2.client.registration.oidc.provider=oidc spring.security.oauth2.client.provider.oidc.authorization-uri=${iam.uri}/oidc/idp/authorize spring.security.oauth2.client.provider.oidc.token-uri=${iam.uri}/oidc/oauth2/token spring.security.oauth2.client.provider.oidc.jwk-set-uri=${iam.uri}/oidc/jwk/keys server.servlet.session.cookie.name=TDPSESSION spring.session.store-type=NONE security.sessions=stateless security.user.password=none |
Single Sign-On security configuration parameters. |
| security.oidc.client.endSessionEndpoint=${iam.uri}/oidc/idp/logout security.oidc.client.logoutSuccessUrl=http://${public.ip}:${server.port} security.oauth2.logout.uri=/signOut security.oauth2.sso.login-path=/signIn iam.scim.url=${iam.api.uri}/scim/ tenant.account.cache.enabled=true |
Single Sign-On properties for Talend Data Preparation API and Gateway. |
|
gateway-api.service.path=/gateway spring.cloud.gateway.mvc.http-client.connect-timeout=5000 |
Single Sign-On configuration parameters. Do not change these values, unless you were told so by Talend. |
| logging.file=data/logs/app.log | Path of the log file storage folder. |
| logging.pattern.level=%5p [user %X{userId}] | Level output pattern for the log file. |
| logging.pattern.file=%d{yyyy-MM-dd HH:mm:ss.SSS} %5p --- [%t] %-40.40logger{39} : %m%n%wEx | Uncomment this parameter to enable log pattern configuration. |
| logging.level.=WARN logging.level.org.talend.dataprep=INFO logging.level.org.talend.dataprep.api=INFO logging.level.org.talend.dataprep.dataset=INFO logging.level.org.talend.dataprep.preparation=INFO logging.level.org.talend.dataprep.transformation=INFO logging.level.org.talend.dataprep.fullrun=INFO logging.level.org.talend.dataprep.api.dataquality=INFO logging.level.org.talend.dataprep.configuration=INFO logging.level.org.talend.dataquality.semantic=INFO |
Talend Data Preparation loggers parameters. |
| logging.pattern.console=%clr(%d{yyyy-MM-dd HH:mm:ss.SSS}){faint} %clr(%5p) %clr(${PID:- }){magenta} %clr(---){faint} %clr([%15.15t]){faint} %clr(%-40.40logger{39}){cyan} %clr(:){faint} %m%n%wEx | Uncomment this parameter to enable console logging pattern configuration. |
| spring.output.ansi.enabled=always | Uncomment this parameter to configure ansi coloration in console output. The possible values are always, never, and detect. |
| logging.config=classpath:logback-spring.xml | Uncomment this parameter to configure the Talend Data Preparation
logging with a custom logback file. Enter the path to your logback file. |
| audit.log.enabled=true talend.logging.audit.config=config/audit.properties |
Audit logs parameters. |
| help.url=https://help.talend.com help.version=content help.facets.version=8.0 help.facets.language=en help.search.url=https://www.talendforge.org/find/api/THC.php help.fuzzy.url=${help.url}/search/all?filters=EnrichPlatform~%2522Talend+Data+Preparation%2522*EnrichVersion~%2522${help.facets.version}%2522&utm_medium=dpdesktop&utm_source= help.exact.url=${help.url}/access/sources/${help.version}/topic?EnrichPlatform=Talend+Data+Preparation&EnrichVersion=${help.facets.version}&utm_medium=dpdesktop support.url=https://www.talend.com/services/technical-support/ community.url=https://community.talend.com/s/group/0F93p0000000VNyCAM/data-quality-preparation-stewardship |
Parameters that allow access to the online documentation. Do not change these values, unless you were told so by Talend. |
| default.text.enclosure=\" default.text.escape=\" default.text.encoding=UTF-8 |
Configurable values for the default enclosure and escape characters for CSV exports, as well as the default text encoding. The default separator can be semicolon ";", tab "\t", space " ", comma "," or pipe "|". |
| default.import.text.enclosure=\" default.import.text.escape= |
Configurable values for the default enclosure and escape characters for CSV imports. |
| app.products[0].id=TDS app.products[0].name=Data Stewardship app.products[0].url=<place_your_tds_url_here> |
When Talend Data Preparation and Talend Data Stewardship are both installed, you have the possibility to switch between the two applications. Configure the URL to your Talend Data Stewardship instance so that you can reach it. |
| dataset.service.provider=legacy management.health.redis.enabled=false management.endpoint.prometheus.enabled=false management.tracing.enabled=false |
Technical parameters. Do not change these values, unless you were told so by Talend. |
| maintenance.scheduled.cron = 0 0 3 * *
* maintenance.scheduled.fixed-delay = 3600000 maintenance.scheduled.initial-delay = 3600000 spring.batch.datasource=mongodb spring.batch.job.enabled=false server.servlet.session.timeout=60m oidc.accessTokenLifetime=3600 features.lineage.enabled = false features.inventory.events.enabled=false dataprep.actions.exclude=detect_outliers dataprep.actions.useDeprecated=true features.items.sharing.key=tdp.sharing.impl features.items.sharing.modes.legacy=legacy features.items.sharing.defaultMode=legacy features.items.sharing.provider=fixed features.statistics.enabled=true |
Maintenance parameters. Do not change these values, unless you were told so by Talend. |
|
spring.main.allow-bean-definition-overriding=true spring.main.allow-circular-references=true |
Properties needed for Spring Boot 2.6. |
dataprep.hybrid.enabled=true client.session-management.enabled=${dataprep.hybrid.enabled} iam.uri=https://iam.us.cloud.talend.comiam.api.uri=https://api.us.cloud.talend.com/v1 iamproxy.service.url=${iam.api.uri}/iam server.portal.url=https://portal.us.cloud.talend.com external.user.preferences.url=${server.portal.url}/user/aboutme security.oidc.client.sessionManagementUri=${iam.uri}/oidc/session-management security.oauth2.client.scope=openid refreshToken entitlements security.xframe.options=DENY |
Uncomment these parameters to configure hybrid mode for Talend Data Preparation after enabling it in the Talend Management Console interface. |
|
server.session.timeout=1800 spring.session.store-type=MONGODB spring.session.timeout=1800 spring.session.mongodb.collection-name=tdp_sessions |
Use these parameters to activate session management when working with two or more Talend Data Preparation instances. |