Skip to main content Skip to complementary content

Invoking a REST API to get streaming UUIDs

Availability-noteDeprecated

This scenario aims at helping you set up and use connectors in a pipeline. You are advised to adapt it to your environment and use case.

Example of a pipeline created from the instructions below.

Procedure

  1. Click Connections > Add connection.
  2. In the panel that opens, select the type of connection you want to create.

    Example

    REST HTTP BIN
  3. Select your engine in the Engine list.
    Information noteNote:
    • It is recommended to use the Remote Engine Gen2 rather than the Cloud Engine for Design for advanced processing of data.
    • If no Remote Engine Gen2 has been created from Talend Management Console or if it exists but appears as unavailable which means it is not up and running, you will not be able to select a Connection type in the list nor to save the new connection.
    • The list of available connection types depends on the engine you have selected.
  4. Select the type of connection you want to create.
    Here, select REST.
  5. Fill in the connection properties and URL address of the REST server to be invoked as described in REST properties, check the connection and click ADD DATASET. In this example, the https://httpbin.org API service returning random universally unique identifiers (UUIDs) is invoked.
    Configuration of a new REST connection.
  6. In the Add a new dataset panel, name your dataset. The REST connector returns a JSON with a parsed body entry.

    Example

    UUID
  7. Fill in the required properties to invoke the service and click View sample to see a preview of your dataset sample:
    1. In the Type field, select Streaming.
    2. In the Resource field, type in uuid to retrieve the uuid records from the https://httpbin.org API service.
    3. In the HTTP method field, select GET.
    4. In the Answer body format field, select JSON.
  8. Click Validate to save your dataset.
  9. Do the same to add the Test dataset that will be used as a destination in your pipeline to receive logs. Fill in the connection properties as described in Test connection properties.
  10. Click Add pipeline on the Pipelines page. Your new pipeline opens.
  11. Give the pipeline a meaningful name.

    Example

    Invoke REST API to get streaming UUIDs
  12. Click ADD SOURCE and select your source dataset, UUID in the panel that opens. The pipeline turns into a streaming pipeline as you have selected the Streaming dataset type.
  13. In the Configuration tab of this source dataset, set the polling interval to stream the API service in the Delay field. By default, it is 5000ms, that is to say data will be retrieved every 5 seconds.
  14. Click the ADD DESTINATION item on the pipeline to open the panel allowing to select the output dataset in which your output logs will be stored.
  15. In the Configuration tab of the destination, enable the Log records to STDOUT option to capture the logs into this output destination.
  16. Click Save to save your configuration.
  17. On the top toolbar of Talend Cloud Pipeline Designer, click the Run button to open the panel allowing you to select your run profile.
  18. Select your run profile in the list (for more information, see Run profiles), then click Run to run your pipeline.

Results

Your streaming pipeline is being executed, the data that was invoked from the REST API service is retrieved and the output logs keep receiving more UUIDs as you refresh them. You can see that UUIDs are retrieved every 5 seconds as previously defined.
Highlight of the UUID retrieved every 5 seconds in the output log.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here