RudderStack’s data plane is responsible for receiving, transforming, and routing event data to the destination. To do so, it receives the event data from sources like websites, mobile apps, server-side applications, cloud apps, and data warehouses.
A simplified version of the RudderStack data plane architecture is as shown:
The following sections give an overview of each of the components of the RudderStack data plane.
The gateway module is primarily responsible for receiving event data from a source (web, mobile, or server-side) and forwarding it for processing and transformation.
It accepts event requests and sends an acknowledgment back to the source depending on the acceptance (HTTP 200 response) or rejection of the event data.
The gateway rejects event data in case of the following scenarios:
Invalid write key
Improper request size
The gateway also temporarily stores all received event data into an internal PostgreSQL database before acknowledging successful receipt. Once event is successfully sent to the destination, it is then deleted from the database.
The processor fetches data from the gateway and forwards it to the transformation module. Once event data is transformed, the processor forwards it to the router module that sends it to the required destination.
Refer to the Transformations guide for more information on user transformations.
Once events are transformed, the transformation module sends them back to the processor. The processor then forwards this data to the router, which in turn relays it to the desired destination.
The router module sends the processed and transformed event data to your desired destinations, like marketing and analytics platforms, CRMs, data warehouses, etc.
The data plane uses PostgreSQL as a streaming database for the event data. RudderStack temporarily stores the events in the database so that it can retry sending them in case of delivery failures. The events are deleted once they are successfully delivered.
RudderStack does not persist any events or store any user information in the database.
Data plane workflow
The following steps detail the RudderStack data plane workflow:
The gateway module receives event data from the source.
The gateway then:
Stores event data in the router database (PostgreSQL).
Sends an HTTP 200 status acknowledging receipt of the data.
The processor module picks the data from the gateway and forwards it to the transformation module.
The transformation module transforms the event and sends it back to the processor.
The processor forwards the transformed event to the router. This event is then deleted from the gateway store.
The router then:
i. Sends the transformed event data to the specified destinations.
ii. Stores the event information in a separate table in the database.
Once the event data reaches the destination, the router deletes it from the database.
Customizing the RudderStack data plane
Although the default configuration works just fine for most use cases, RudderStack also gives you the flexibility to customize the data plane with a variety of configuration options. Some of these include backing up events to S3, rejecting malicious requests by defining the maximum size of the event, and more.
You can do so by editing the config.yaml file to suit your application’s needs.
cookies, the cookies that are categorized as necessary are stored on your browser as they are as
for the working of basic functionalities of the website. We also use third-party cookies that
analyze and understand how you use this website. These cookies will be stored in your browser
consent. You also have the option to opt-out of these cookies. But opting out of some of these
have an effect on your browsing experience.
Necessary cookies are absolutely essential for the website to function properly. This
category only includes cookies that ensures basic functionalities and security
features of the website. These cookies do not store any personal information.
learn more about cookies and why we use them, visit our cookie
policy. We'll assume you're ok with this, but you can opt-out if you wish Cookie Settings.