Event Replay
- growth
- enterprise
4 minute read
RudderStack’s Event Replay feature lets you back up your event data and replay it in case of any failures. You can leverage this feature in the following scenarios:
- Replay failed events from a particular date to a destination, maybe due to some misconfiguration.
- Replay all events for a particular source, from a particular date, to a new event stream or warehouse destination.
Feature support by plan
- Free plan users cannot replay events.
- Growth plan users can replay events from up to 7 days depending on their event data retention options.
- Enterprise plan users can replay events from up to 30 days depending on their event data retention options.
Supported event types for replay
RudderStack does not support replaying data from Reverse ETL sources.
RudderStack can store the following types of event data and replay it to the specified destinations:
Raw events
RudderStack stores the raw events captured through various sources in batches of 100,000 events.
Note the following:
- Upload frequency: The upload frequency is 100,000 events or 5 minutes (whichever is earlier) for each source.
- File format: RudderStack stores the events in newline-separated gzipped JSON format.
Delivery failures
RudderStack stores only the metadata of the delivery failures in its own S3 storage, approximately every 30 seconds.
RudderStack does not store any event payloads in case of delivery failures, as the payloads are already stored during archival.
Event data retention options
Event data retention options are available in the Growth and Enterprise plans.
Configure them in Settings > Workspace > Data Management. See Data Management for details.
RudderStack offers the following event data retention options for event replay:
| Data retention option | Description | Recommended for replay? |
|---|---|---|
| Do not store event data | RudderStack does not store any event data and hence it cannot be replayed. | |
| Store in your own cloud storage | Event data is stored in your object storage. RudderStack currently supports Amazon S3, Google Cloud Storage, Azure Blob Storage, and MinIO as the object storage options. | |
| RudderStack 7-day cloud storage | This is the default setting on Growth and Enterprise plans — RudderStack stores event data on a rolling 7-day basis to prevent data loss if a rare incident occurs. | |
| RudderStack 30-day cloud storage | RudderStack stores the event data in its S3 bucket on a rolling 30-day basis, that is, RudderStack deletes the data 30 days after it was stored. Available in the Enterprise plan only. |
Object storage setup
If you choose to set up your own object storage for backing up and replaying the events, follow the steps below depending on your cloud provider:
The S3 setup for event backup and replay is different to setting up Amazon S3 as a destination.
- Create your object storage bucket.
- Configure the relevant permissions for your bucket.
- Connect your storage provider in the RudderStack dashboard. See Store event data in your own cloud storage for details.
- Create your object storage bucket.
- Configure the relevant permissions for your bucket.
- Connect your storage provider in the RudderStack dashboard. See Store event data in your own cloud storage for details.
- Login to the Azure portal and create a storage account.
- Click Containers under Blob service and create a new container.
- Connect your storage provider in the RudderStack dashboard. See Store event data in your own cloud storage for details.
- Log in to your MinIO service and set up your bucket.
- Connect your storage provider in the RudderStack dashboard. See Store event data in your own cloud storage for details.
Event ordering behavior
RudderStack processes replayed events in the same order they were originally received at the source.
Important consideration
While RudderStack processes events in their original order, destinations may overwrite newer data with older replayed data, depending on how each destination handles events. Consider this behavior when replaying your events to avoid unintended data overwrites.
How can I replay my events?
Depending on how you store your event data (RudderStack-hosted storage or your own object storage), contact RudderStack Support to safely replay your events and avoid any unfavorable circumstances.
