Syncing Events

RudderStack supports syncing data from the supported Reverse ETL sources to the downstream destinations as identify and track calls.

You can send your data as track calls only while mapping your warehouse columns to the destination fields using the JSON builder. Visual Data Mapper is not supported currently.

Key features

Some key features of syncing events from Reverse ETL sources are listed below:

• Ensure the reliability of your events: Sometimes, your critical events(such as purchases) may get blocked by things outside your control, such as adblockers. In such cases, if you have all your users’ activity in the database, you can send it to the downstream systems as a guaranteed source of truth.
• Simulate historical tracking of events: You can replay the historical data from other data sources to your sales, support, and marketing systems to maintain business continuity.
• Fix bad data and replay it: There is a possibility that your destination ends up with some bad data due to a bug in the tracking code or any other reason. In such cases, you can clear the bad data and replay a clean version by syncing the events.
• Test your transformations and integrations: You can use a sample dataset to test an event stream for a new version of your transformation or a new integration using the track call.

Use cases for sending track events

Some examples where you can use the track call to send events from your warehouse into the downstream destinations are listed below:

• Sending event data to engagement platforms such as Braze, Iterable, Customer.io, or Salesforce Marketing Cloud to trigger behavior or populate user actions.
• Sending event data to the downstream systems that are configured to handle track calls, like Apache Kafka or Amazon Kinesis.
• Replaying the fact table that might be the source of truth for user behavior to the downstream tools such as Statsig, Optimizely, etc.

Steps for syncing data

To sync your data from a Reverse ETL source to a downstream destination, follow these steps:

1. Set up your Reverse ETL source.
2. Connect the source to the desired destination by configuring the destination-specific settings.
3. Based on how you want to ingest the data from the source, follow the steps listed in either of the following guides:

• Sync data from a warehouse table: If you have configured your source using the Table option.
• Sync data from a model: If you have configured your source using the Model option.
• Sync data from an audience : If you have configured your source using the Audience option.

Start and stop syncs

To start a new sync, go to the Syncs tab of your reverse ETL source and click Sync Now.

When you sync data from the source for the first time, RudderStack performs a a full sync, that is, it syncs all of the data from the source.

It then incrementally syncs any new data since the last sync whenever you press the Sync Now button.

To force a full data sync, choose the Force Full Resync option.

To stop a running sync, click the Stop Now button.

In the confirmation popup, click Stop Sync to cancel and stop the sync.

Note the following:

• Once you stop a sync, any data that RudderStack reads from the warehouse and is on the fly may not be stopped or dropped from being delivered to the destination. RudderStack only stops reading and sending any new data from the warehouse to the destination and prevents the sync from progressing.
• For a very low number of deltas (new data since the last attempted sync), you may sometimes see a “0 deltas succeeded” after you stop the sync.
• Once you cancel a sync, the behavior of the next sync depends on the type of the cancelled sync. For example, if a cancelled sync is an incremental sync, then the next sync will be incremental too. Similarly, if the cancelled is a full sync, then the next sync will be a full sync.
• RudderStack makes sure there is no data loss in the next sync as a result of the previous cancelled sync irrespective of the sync type (full/incremental) or sync mode(upsert/mirror).

Drain behavior on sync cancellation

This process is also applicable for scenarios where a Reverse ETL source is disabled or disconnected from a destination.

Whenever you stop a Reverse ETL sync, there might be some events that are in the process of being delivered and need to be drained from the pipeline.

This is important to make sure the next sync is not affected and you don’t have to wait for the events already in pipeline to be delivered or failed.

After you stop a sync, RudderStack automatically triggers this draining process in the background. The time taken for this process to complete depends on the amount of data. Hence, waiting for some time before triggering next sync is recommended.

Questions? Contact us by email or on Slack