Skip to main content Skip to complementary content

Setting up the Job

Procedure

  1. Double-click tSnowflakeConnection to open its Basic settings view and enter all the information that is needed to establish the Snowflake connection.
  2. Double-click tFixedFlowInput_1 to open its Basic settings view and do the following.
    1. Click [...] next to Edit schema and add these columns to the schema: firstname, type String; lastname, type String; and dept, type String.
    2. Select Use Inline Table and enter the following in the Inline Table field. 
      "Andrew";"Wallace";"Doc"
"John";"Smith";"R&D"
"Christian";"Dior";"Sales"
    3. Leave the other options as they are.
    This component defines the record fields and creates three records for the scenario.
  3. Double-click tFileOutputParquet_1 to open its Basic settings view and do the following.
    1. Click the Sync columns button to make sure the schema of this component has the same columns as those of the tFixedFlowInput component.
    2. Enter the path to the Parquet file which you want to write the data into in the File name field.
      You can also enter the file path in this field by clicking the [...] button next to this field and navigating to the desired Parquet file.
    3. Select the action you want to take to the file by selecting an item from the Action drop-down list.
      If the file specified in the File name field does not exist, select Create from this drop-down list; otherwise, select Overwrite.
    4. Leave the other options as they are.
    This component writes the data to a specified file.
  4. Double-click tFileInputParquet_1 to open its Basic settings view and do the following.
    1. Click the [...] button next to Edit schema to open the schema editor and add the same three columns as those of the tFileOutputParquet component.
    2. Enter the path to the file specified for the tFileOutputParquet component in the File name field.
      You can also fill the global variable that references the file path in this field by placing the cursor in this field, pressing Ctrl + Space, and selecting tFileOutputParquet_1.FILE_PATH from the context menu.
    3. Leave the other options as they are.
    This component retrieves the data from the file which the tFileOutputParquet_1 writes data into and passes the data to the subsequent component (that is, tSnowflakeOutputBulkExec).
  5. Double-click tSnowflakeOutputBulkExec to open its Basic settings view and do the following.
    1. Select tDBConnection_1 - tSnowflakeConnection from the Connection Component drop-down list.
    2. Click the [...] button next to the Table field and select a table from the list box in the dialog box that appears.
      The data will be loaded into this table. Tables in the list box are existing tables. To load the data into a new table, select the Use custom object option in the dialog box and enter the name of the new table in the Object Name field.
    3. Click the Sync column button and then check that the component has the same columns in the schema as those of the tFileInputParquet_1 component.
    4. Select the action you want to take to the selected table from the Table Action drop-down list.
    5. Select the action you want to take for the data received from the Output Action drop-down list.
    6. Select Internal from the Storage drop-down list to store the received data in the internal storage.
    7. Enter the name of the folder where the received data will be stored in the Stage Folder field.
    8. Leave the other options as they are.

    This component receives data from the tFileInputParquet component and loads the data into a specified Snowflake table.

  6. Double-click tSnowflakeInput to open its Basic settings view and do the following.
    1. Select tDBConnection_1 - tSnowflakeConnection from the Connection Component drop-down list.
    2. Click the [...] button next to the Table field and select the same table as the one selected for the tSnowflakeOutputBulkExec component.
    3. Click the [...] button next to Edit schema to open the schema editor and add the same three columns as those of the tFileOutputParquet component.
    4. Leave the other options as they are.
    This component retrieves the data from the Snowflake table specified for the tSnowflakeOutputBulkExec component.
  7. Double-click tLogRow_1 to open its Basic settings view and do the following.
    1. Click the Sync column button and then check that the component has the same columns in the schema as those of the tSnowflakeInput component.
    2. Select Table (print values in cells of a table).
    3. Leave the other options as they are.
    This component verifies the data that is written into the Parquet file and then loaded to the Snowflake table.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here