RudderStack's Transformations feature enables you to write custom JavaScript functions to implement specific use-cases on your event data, such as:

• Filtering or sampling events
• Cleaning or aggregating data
• Data masking or removing sensitive PII to ensure data privacy
• Enriching events by implementing static logic or leveraging an external API
• Using an API to implement specific actions on the events

You can use this feature across your Cloud Apps, Reverse ETL, and Cloud Extract pipelines.

Key features

Some key features of RudderStack transformations are listed below:

• You can apply the transformations to your events in real-time.
• You can programmatically add and remove transformations using the Transformations API.
• They're easier to build, manage, and debug, and reuse.
• You can create an organization-wide sandbox where your team can store the transformations before publishing them in a production environment.
• You can version control your transformations.

How do transformations work?

At a high level, once you add a transformation and connect it to a destination, RudderStack:

1. Tracks and collects events at the source
2. Applies the transformation logic to the your events
3. Forwards the transformed event to your connected destination

Use case

Suppose you want to set the context.os field to Android for all the events, irrespective of the actual platform RudderStack tracks the event from. You can write a simple transformation to do this:

Copy
export function transformEvent(event, metadata) {
    event.context.os = { name: "Android"};
    return event;
}

The transformEvent function overrides the event's context.os.name and sets it as Android, as seen below:

Adding a transformation

To add a new transformation in the RudderStack dashboard, follow these steps:

1. Log into the RudderStack dashboard.
2. Go to Enhance > Transformations, as shown:

3. Click New Transformation.
4. Add a name for your transformation and an optional description.
5. Next, add your JavaScript function in the Transformation window, as shown:

6. To test your transformation, paste your sample event in the Events block and click Run Test. By default, RudderStack provides some sample events to test if your transformation logic works as expected.

   

7. To save the transformation, click Save.

Debugging with logs

RudderStack enables you to capture any event-related information in the form of logs while running a test on your transformation. You can do this by including the log function in your transformation code.

An example of using the log function is shown below:

Copy
export function transformEvent(event, metadata) {
  const meta = metadata(event);
  event.sourceId = meta.sourceId;

  log("Event Name is", event.event, ";", "Message ID is", event.messageId);
  log("Source ID is", meta.sourceId);

  return event;
}

On adding the above transformation and clicking Run Test, you can see the resulting log in the Logs section of the dashboard, as shown:

Connecting transformation to a destination

You can connect a transformation to a destination in two cases:

Case 1: While setting up a new destination

RudderStack provides the option to connect an existing transformation or create a new transformation while setting up a destination, as shown:

Case 2: While connecting to an existing destination

To add a transformation to an existing destination, follow these steps:

1. In the dashboard, go to the Transformation tab in your destination dashboard and click Add a transformation, as shown:

2. Then, choose the transformation to connect to the destination.

Deleting a transformation

To delete a transformation, go to Enhance > Transformations and click the Delete button next to the transformation that you want to delete, as shown:

transformEvent function

While using a transformation, RudderStack applies the transformEvent function on each event that takes two arguments:

• event: Corresponds to the input event.
• metadata (optional): Corresponds to the JavaScript function which you can use to access the metadata of the given event.

After the transformation is complete, transformEvent returns the final event to be sent to the destination.

Accessing event metadata

RudderStack injects a function metadata(event) into your transformations as an argument. This allows you to access the event metadata variables that help you customize your transformations.

The following properties, if available, are present in the metadata response:

An example of using metadata is shown below:

Copy
export function transformEvent(event, metadata) {
  const meta = metadata(event);
  event.sourceId = meta.sourceId;

  return event;
}

Applying transformation on a batch of events

You can also perform any aggregation or roll-up operation on a batch of events using the transformBatch function instead of transformEvent, as shown:

Copy
export function transformBatch(events, metadata) {
    return events;
}

Libraries

One of the key features of RudderStack transformations is the ability to reuse the transformation code in other transformations using the Libraries feature.

Adding a library

To add a new library, follow these steps:

1. In the RudderStack dashboard, go to Enhance > Transformations and click the Libraries tab, as shown:

2. Click New library.
3. Add the library Name, Description and write the function you want to reuse across different transformations. You can also add multiple functions in a single library, as shown:

4. Click Run Test to ensure the library code has the correct syntax.

Using libraries in transformations

To use the libraries in your existing transformations, refer to the Import Library Name option in the RudderStack dashboard, as shown:

You can then use the library in a transformation with a simple import statement. Refer to the below use case for more information.

Use case

Suppose you want to import a function rudderEmail from the Is Rudder Email library to filter events that don't have the email address containing the RudderStack domain.

The rudderEmail function is as follows:

Copy
export function rudderEmail(email) {
    return /@(rudderlabs|rudderstack)| \+ruddertest/.test(email);
}

The following code snippet demonstrates how you can implement this in a transformation:

Copy
import { rudderEmail } from "isRudderEmail";
export function transformEvent(event) {
  const email =
    event.context && event.context.traits && event.context.traits.email;
  if (email) {
    if (!rudderEmail(email)) return;
  }
  return event;
}

On clicking Run Test, a sample event not containing the RudderStack email domain is filtered out, as shown:

Importing multiple functions from a single library

The following snippets highlight how to properly import functions from a library:

Copy
// ---------------
import { getLoss } from "getFinanceData";

// OR

import { getLoss, getRevenue, getProfit } from "getFinanceData";
import {
  getLoss,
  getRevenue,
  getProfit
} from "getFinanceData";

// For default getPrice import
import getPrice, { getRevenue, getProfit } from "getFinanceData";

// alias imports
import getPrice as gp, { getRevenue as gr, getProfit } from "getFinanceData";
// usage: gp(event), gr(event), getProfit(ev)

import * as GFD from "getFinanceData";
// usage: GFD.getRevenue(ev), GFD.getProfit(ev)
// for default import: GFD.default(ev)

The following snippets highlight the incorrect way of importing the library functions:

Copy
// -----------------
import * from "getFinanceData";
getPrice(ev)

// OR

import getPrice as gp from "getFinanceData";
getPrice(ev)

Deleting a library

To delete a transformation, go to Enhance > Transformations and click the Libraries tab. button next to the transformation that you want to delete, as shown:

Making external API requests

You can make any number of external API requests in your transformation functions and use the response to enrich your events.

RudderStack injects an asynchronous fetch function in your transformations. It makes an API call to the given URL and returns the response in the JSON format.

You can use the fetch function in your transformations, as shown:

Copy
export async function transformEvent(event, metadata) {
  const res = await fetch("post_url", {
    method: "POST",  // POST, PUT, DELETE, GET, etc.
    headers: {
      "Content-Type": "application/json;charset=UTF-8",
      Authorization: "Bearer <authorization_token>"
    },
    body: JSON.stringify(event)
  });
  event.response = JSON.stringify(res);
  return event;
}

Fetching response properties using fetchV2

FetchV2 is a wrapper for the fetch call. It enables you to fetch the response properties more efficiently while making the external API calls.

The following properties are present in a fetchV2 response:

The below example highlights the use of the fetchV2 function in a transformation to capture failure due to a timeout:

Copy
export async function transformEvent(event) {
  try {
    const res = await fetchV2("url", { timeout: 1000});
    if (res.status == 200) {
      event.response = JSON.stringify(res.body);
    }
  } catch (err) {
    log(err.message);
  }
  return event;
}

Limits

You should take into account the following memory and time limits when invoking a transformation:

FAQ

For answers to some of the commonly asked questions on this feature, refer to the FAQ guide.