End-to-End Walkthrough: Transformations Management with Rudder CLI Beta

In this tutorial, you will use the Rudder CLI tool to define, validate, test, and apply transformations from your local project.

Prerequisites

• Rudder CLI tool (rudder-cli) installed locally
• A project directory with your transformation YAML files
• In your RudderStack workspace, create a workspace-level Service Access Token with the following permissions:

• If you’re on Free or self-hosted plan, or for testing and development only: Generate a Personal Access Token with Read-Write role

Any action authenticated by a Personal Access Token will break if the user generating the token is removed from the organization or there is a breaking change to their permissions.

Token permissions for legacy RBAC system

If you are on the legacy Permissions Management (RBAC) system, your workspace-level Service Access Token should have the Admin role and Grant edit access toggled on under Transformations.

See Generate a workspace-level Service Access Token for steps to create the token.

1. Authenticate the CLI tool

Run the following command and enter your access token when prompted:

rudder-cli auth login

2. Create a project directory

Create a project directory to store your transformation specs, code, and tests:

mkdir -p ~/tutorial-transformations/transformations

Example structure:

tutorial-transformations/
└── transformations/
    ├── my-transformation.yaml
    ├── my-library.yaml
    ├── my-python-transformation.yaml
    ├── my-python-library.yaml
    ├── javascript/
    │   ├── my-transformation.js
    │   └── my-library.js
    ├── python/
    │   ├── my-python-transformation.py
    │   └── my-python-library.py
    └── tests/
        ├── input/
        │   └── product_clicked.json
        └── output/
            └── product_clicked.json

RudderStack supports Python transformation libraries only in the Growth and Enterprise plans.

3. Add a transformation spec

This section lists the steps to create a transformation YAML spec via the following methods:

• Referencing an external file
• Using inline code

Referencing an external file

1. Create the YAML spec for your transformation:

version: rudder/v1
kind: transformation
metadata:
  name: my-transformation
spec:
  id: my-transformation
  name: My Transformation
  description: Add static metadata to each incoming event
  language: javascript
  file: javascript/my-transformation.js
  tests:
    - name: Basic payload test
      input: tests/input
      output: tests/output

version: rudder/v1
kind: transformation
metadata:
  name: my-python-transformation
spec:
  id: my-python-transformation
  name: My Python Transformation
  description: Tag each incoming event with a processing timestamp
  language: python
  file: python/my-python-transformation.py
  tests:
    - name: Basic payload test
      input: ./tests/input
      output: ./tests/output

2. Create the transformation code file:

export function transformEvent(event, metadata) {
  event.context = event.context || {};
  event.context.cliManaged = true;
  return event;
}

from datetime import datetime, timezone

def transformEvent(event, metadata):
    event.setdefault("context", {})
    event["context"]["processedAt"] = datetime.now(timezone.utc).isoformat()
    return event

Using inline code

You can also write transformation code directly in the YAML spec using code instead of referencing an external file with file:

version: rudder/v1
kind: transformation
metadata:
  name: inline-transformation
spec:
  id: inline-transformation
  name: Inline Transformation
  description: Add processing metadata to each event
  language: javascript
  code: |
    export function transformEvent(event, metadata) {
      event.context = event.context || {};
      event.context.processedBy = "inline-transformation";
      return event;
    }    

version: rudder/v1
kind: transformation
metadata:
  name: inline-python-transformation
spec:
  id: inline-python-transformation
  name: Inline Python Transformation
  description: Add processing metadata to each event
  language: python
  code: |
    def transformEvent(event, metadata):
        event.setdefault("context", {})
        event["context"]["processedBy"] = "inline-python-transformation"
        return event    

code and file are mutually exclusive — use one or the other in a given spec. See the Transformation YAML Reference for the complete schema.

4. Add a transformation library spec

Create the YAML spec for your transformation library:

version: rudder/v1
kind: transformation-library
metadata:
  name: my-library
spec:
  id: my-library
  name: my library
  description: Shared helper utilities
  language: javascript
  file: javascript/my-library.js
  import_name: myLibrary

version: rudder/v1
kind: transformation-library
metadata:
  name: my-python-library
spec:
  id: my-python-library
  name: my python library
  description: Shared Python helper utilities
  language: python
  file: python/my-python-library.py
  import_name: myPythonLibrary

Then, create the library code file:

export function addTag(event, key, value) {
  event.context = event.context || {};
  event.context[key] = value;
  return event;
}

def add_tag(event, key, value):
    event.setdefault("context", {})
    event["context"][key] = value
    return event

For library resources, import_name must be the camelCase form of name. For example, name: my library maps to import_name: myLibrary and name: my python library maps to import_name: myPythonLibrary.

5. Validate YAML and code references

Run validation from the project root:

rudder-cli validate -l ~/tutorial-transformations

This command validates required fields, code/file rules, supported languages, code syntax, and test definitions.

6. Test transformations locally

Rudder CLI supports 3 test modes:

You cannot include multiple test modes in a single command.

1. Test a specific transformation by ID:

rudder-cli transformations test my-transformation -l ~/tutorial-transformations

2. Test all transformations:

rudder-cli transformations test --all -l ~/tutorial-transformations

3. Test only new or modified transformations:

rudder-cli transformations test --modified -l ~/tutorial-transformations

Optional

• Show detailed failures and diffs:

rudder-cli transformations test --all --verbose -l ~/tutorial-transformations

• Show the built-in default events:

rudder-cli transformations show-default-events

How it works

When running tests, Rudder CLI loads input files from each test suite’s input path. If a matching output file (same filename) exists in the output path, it compares the actual output against the expected output. If no matching output file exists, the test still runs but skips the output comparison.

Rudder CLI tests the transformation using default RudderStack events if:

• spec.tests isn’t configured, or
• No input JSON files are found in the configured input paths

See the Rudder CLI transformations testing framework section for more details.

7. Apply changes

1. Optional dry-run:

rudder-cli apply -l ~/tutorial-transformations --dry-run

2. Apply to workspace:

rudder-cli apply -l ~/tutorial-transformations

Next steps

• See the Transformation YAML Reference for detailed resource schemas and constraints
• See Manage Transformations Using Rudder CLI for the overall CLI workflow
• Automate with GitHub Actions