Import data using tables

This guide is applicable for Reverse ETL sources configured using the Table option. If you have configured your Reverse ETL source using the Model option, refer to the Importing Data using Models guide.

RudderStack provides the following options to map your warehouse columns to specific destination fields while importing the data:

• Map with Visualizer (Refer to the Visual Data Mapper guide for the list of the supported destinations.)
• Map with JSON

This guide lists the JSON mapping settings required to import and sync data from your warehouse table to the specified destination.

Data import settings

The settings to import and sync data from your warehouse table are listed below:

• Schema: Specify your warehouse schema.
• Table: Specify the table residing in the above schema from which you want RudderStack to import the data.
• Sync mode: Select the sync mode that RudderStack should use to sync your data. For more information on these modes, refer to the Sync Modes guide.
• Primary Key: Select a column from the table specified above to uniquely identify your records in the warehouse.

RudderStack uses the primary key column for diffing in case of incremental syncs. You can generate it using either of the following ways:

• Generating your table with a primary key OR
• Creating a table view The best combination for a primary key is the timestamp and user_id.

• Event Type: Select from the identify or track event type depending on how you want to send the event data to the downstream destinations. If you select track, you also need to provide:

  • Event Name: Enter the event name which should be sent for all events to the downstream destinations:

  

  You can also send different event names by enabling the lookup event name by column setting and specifying the column name which should be used to set the event name in the Event Name field:

  

Refer to the Syncing Events guide for more information on sending the event data using the identify or track call.

• Choose user identifier: Choose atleast one user identifier from user_id and anonymous_id from the dropdown.

You can also preview the data snippet which RudderStack will send to the destination. All the columns from the table are selected by default. However, you can choose to retain specific columns by searching and selecting/deselecting them. Finally, preview the resulting JSON on the right:

Here, you can select all or only specific columns of your choice, search the columns by a keyword, and also edit the JSON Trait Key. You can also preview the resulting JSON on the right.

The JSON payload carries the user_id and anonymous_id from the columns selected in the Choose user identifier section. Moreover, the traits are used from the columns selected in the Column section.

Add Constant

You can also use the Add Constant option to add a constant key-value pair which is always sent in the JSON payload:

The new constant will appear in the table and also in the JSON preview inside the traits, as shown:

You can also use the dot notation to define a constant, as shown:

Once you have finalized the configuration, click Save.

Updating existing configuration

1. Go to the Schema tab of your configured source.
2. Click the Update button, as shown:

3. Update your column selection.

When updating the configuration, you can only change the existing mappings. The Schema, Table, Sync mode, and the User identifier fields are not editable.

4. Finally, click the Save button.

After updating the configuration, the next sync will be a full sync.

Questions? Contact us by email or on Slack