Skip to main content Skip to complementary content

Generating a REST call using input records

Availability-noteDeprecated

This scenario aims at helping you set up and use connectors in a pipeline. You are advised to adapt it to your environment and use case.

Example of a pipeline created from the instructions below.

Procedure

  1. Click Connections > Add connection.
  2. In the panel that opens, select the type of connection you want to create.

    Example

    REST destination
  3. Select your engine in the Engine list.
    Information noteNote:
    • It is recommended to use the Remote Engine Gen2 rather than the Cloud Engine for Design for advanced processing of data.
    • If no Remote Engine Gen2 has been created from Talend Management Console or if it exists but appears as unavailable which means it is not up and running, you will not be able to select a Connection type in the list nor to save the new connection.
    • The list of available connection types depends on the engine you have selected.
  4. Select the type of connection you want to create.
    Here, select REST.
  5. Fill in the connection properties and URL address of the REST server to be called as described in REST properties, check the connection and click Add dataset. In this example, the https://requestinspector.com API service is used to post a record with specific query parameters.
  6. In the Add a new dataset panel, name your dataset.

    Example

    Request Inspector API
  7. Fill in the required properties to connect to the service:
    1. In the Type field, select Batch.
    2. In the Resource field, type in inspect/talend-doc/dataset/{/entity}/{/id} (where talend-doc is the endpoint name that has been created previously on Request inspector website).
    3. In the HTTP method field, select POST.
    4. In the Answer body format field, select Text.
    5. Enable the Set header option and enter header1 in the Name field and header/{/val_head1} in the Value field.
    6. Enable the Set URL parameters option and enter q in the Name field and query_{/q} in the Value field.
    7. Enable the Set path parameters option and enter entity in the Name field and {/entity/type} in the Value field. Add another path parameter and enter id in the Name field and {/entity/book/id} in the Value field.
    8. Enable the Define body option and select JSON in the Body type list and enter the following in the JSON field: 
      {
book:{
"name": "${/entity/book/title}",
"price": ${/entity/book/price}
}
}
    Configuration of a new REST dataset.
  8. Click Validate to save your dataset.
  9. Add a Test dataset that will be used as a source in your pipeline to post a fake record to the REST API destination. Fill in the connection properties as described in Test connection properties and fill in the dataset properties as follows:
    1. Name the dataset Fake record.
    2. Select JSON in the format list.
    3. Enter the following code in the values field: 
      {
"q": 12345,
"val_head1": "head1",
"entity":{
"type": "book",
"book": {
"id": "A321A32A13A2",
"title": "Twenty Thousand Leagues Under the Sea",
"price": 5.60
}
}
}
  10. Click Add pipeline on the Pipelines page. Your new pipeline opens.
  11. Give the pipeline a meaningful name.

    Example

    Generate a REST call using input records
  12. Click ADD SOURCE and select your source dataset, Fake record in the panel that opens.
    Data sample of the fake record.
  13. Click the ADD DESTINATION item on the pipeline to open the panel allowing to select the output REST dataset which will allow you to post the record to the URL defined on the https://requestinspector.com/ REST API.
    Information noteNote: The REST connector sends a request for each record received, so if you had 10 records, you would generate 10 REST requests.
  14. Click Save to save your configuration.
  15. On the top toolbar of Talend Cloud Pipeline Designer, click the Run button to open the panel allowing you to select your run profile.
  16. Select your run profile in the list (for more information, see Run profiles), then click Run to run your pipeline.

Results

Your pipeline is being executed, the fake record has been posted to the REST API service according to the URL parameters you have defined in your REST dataset configuration.

You can then check the result of the call on the REST API and see that the query parameters, the header as well as the body of the JSON record have been received on the https://requestinspector.com/ dashboard:
Highlight of the query parameters, the header and the body of the JSON record on the Request Inspector.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here