Skip to main content Skip to complementary content
  • New archived content: Talend MDM, Talend Data Catalog 8.0, and Talend 7.3 products reached their end of life in 2024. Their documentation was moved to the Talend Archive page and will no longer receive content updates.
Close announcements banner

Exporting logs to an external folder

You can export the engine logs to a dedicated folder for a third-party tool to handle the log files.

You need to maintain and clean up this folder by yourself.

Procedure

  1. Open the <RemoteEngineInstallationDirectory>/etc/org.ops4j.pax.logging.cfg file and add the following lines:

    Example

    # Activate the External folder appender.
    # Be sure to configure/change the MY_EXTERNAL_FOLDER to have logs at the place you want.
    log4j2.rootLogger.appenderRef.ExternalFolder.ref = ExternalFolder
    # External folder appender
    log4j2.appender.external.type = Routing
    log4j2.appender.external.name = ExternalFolder
    log4j2.appender.external.rewritePolicy.type = TalendKarafRewritePolicy
    log4j2.appender.external.routes.type = Routes
    log4j2.appender.external.routes.pattern = $\\{ctx:flowExecutionId\\}
    log4j2.appender.external.routes.route1.type = Route
    log4j2.appender.external.routes.route1.rolling.type = RollingFile
    log4j2.appender.external.routes.route1.rolling.name = flowexecution-$\\{ctx:flowExecutionId\\}
    log4j2.appender.external.routes.route1.rolling.fileName = MY_EXTERNAL_FOLDER/$\\{ctx:flowID\\}/$\\{ctx:flowExecutionId\\}.log
    log4j2.appender.external.routes.route1.rolling.filePattern = MY_EXTERNAL_FOLDER/$\\{ctx:flowID\\}/$\\{ctx:flowExecutionId\\}.log.%i
    # Use the 3 lines below for json pattern
    log4j2.appender.external.routes.route1.rolling.layout.type = JsonTemplateLayout
    log4j2.appender.external.routes.route1.rolling.layout.eventTemplateUri=${karaf.base.uri}/etc/jsonLogMinTemplate.json
    log4j2.appender.external.routes.route1.rolling.layout.stackTraceElementTemplateUri=${karaf.base.uri}/etc/StackTraceElementLayout.json
    # Uncomment two lines below for text pattern 
    #log4j2.appender.external.routes.route1.rolling.layout.type = PatternLayout
    #log4j2.appender.external.routes.route1.rolling.layout.pattern = ${log4j2.pattern}
    log4j2.appender.external.routes.route1.rolling.policy.type = SizeBasedTriggeringPolicy
    log4j2.appender.external.routes.route1.rolling.policy.size = 100MB
    log4j2.appender.external.routes.route1.rolling.strategy.type = DefaultRolloverStrategy
    log4j2.appender.external.routes.route1.rolling.strategy.max = 1
    # By having this set to ${ctx:flowExecutionId} it will match when flowExecutionId is not set in the context
    log4j2.appender.external.routes.route2.type = Route
    log4j2.appender.external.routes.route2.key = $\\{ctx:flowExecutionId\\}
    log4j2.appender.external.routes.route2.null.type = CountingNoOp
    log4j2.appender.external.routes.route2.null.name = CountingNoOp
    This code is only an example. When you copy it to use it in your system, do not forget to change MY_EXTERNAL_FOLDER to the actual directory in which you want to store the logs, for example, /home/usr/my-re-log on Linux or D:/remote-engine/my-log-folder on Windows.

    The $\\{ctx:flowID\\}/$\\{ctx:flowExecutionId\\} part represents the variables to be used to name the log files. If you do not need a dynamically generated log file name, replace them with plain text.

    All these changes make the log4j2.appender.external.routes.route1.rolling.fileName and the log4j2.appender.external.routes.route1.rolling.filePattern lines look like:
    • On Windows:
      log4j2.appender.external.routes.route1.rolling.fileName = C:/my-personal-folder/my-log-foler/my-log.log
      log4j2.appender.external.routes.route1.rolling.filePattern = C:/my-personal-folder/my-log-foler/my-log.log.%i
    • On Linux:
      log4j2.appender.external.routes.route1.rolling.fileName = /home/usr/my-re-log/my-log.log
      log4j2.appender.external.routes.route1.rolling.filePattern = /home/usr/my-re-log/my-log.log.%i
  2. Save the file.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!