Dev-Tools: Automate Transformations Management with Rudder CLI and GitLab CI/CD

Mon, 01 Jan 0001 00:00:00 +0000

This guide explains how to validate, test, and manage your RudderStack transformations directly via GitLab CI/CD pipelines using the Rudder CLI CI/CD components.

Overview

By leveraging the Rudder CLI GitLab CI/CD components, you can:

Validate: Check your transformation YAML configurations for syntax and structure.
Test: Execute transformation code against test events using the specialized transformations-test component.
Apply: Deploy your transformations and libraries to your RudderStack workspace automatically.

Prerequisites

A GitLab repository containing your Rudder CLI project files (transformations and libraries).
A workspace-level Service Access Token with the following permissions:

Resource	Permissions
Transformations	Edit, Connect, Create & Delete
Transformation Libraries	Edit

For testing or development purposes only: Generate a Personal Access Token with Read-Write role

RudderStack recommends using a workspace-level Service Access Token for authentication.

Any action authenticated by a Personal Access Token will break if the user is removed from the organization or a breaking change is made to their permissions.

Token permissions for legacy RBAC system

If you are on the legacy Permissions Management (RBAC) system, your workspace-level Service Access Token should have the Admin role and Grant edit access toggled on under Transformations.

See Generate a workspace-level Service Access Token for steps to create the token.

Setup

Follow these steps to set up the GitLab CI/CD pipeline for your transformations.

Step 1: Configure CI/CD variables

In your GitLab project, go to Settings > CI/CD > Variables.
Add a new variable:
- Key: RUDDERSTACK_ACCESS_TOKEN
- Value: The Service Access Token generated in the Prerequisites section.
- Type: Variable
- Protect variable: Recommended for production branches
- Mask variable: Recommended to hide the token in logs

Step 2: Create GitLab CI/CD configuration

Create or update your .gitlab-ci.yml file in the root of your repository:

stages: [validate, test, deploy]

include:
 # Validate transformation configurations
 - component: $CI_SERVER_FQDN/rudderstack/components/rudder-cli/validate@<VERSION>
 inputs:
 location: "project/"
 access_token: $RUDDERSTACK_ACCESS_TOKEN

 # Test modified transformations in merge requests
 - component: $CI_SERVER_FQDN/rudderstack/components/rudder-cli/transformations-test@<VERSION>
 inputs:
 location: "project/"
 access_token: $RUDDERSTACK_ACCESS_TOKEN
 applies_to: "modified"
 verbose: true
 rules:
 - if: $CI_PIPELINE_SOURCE == "merge_request_event"

 # Apply changes when merged to the default branch
 - component: $CI_SERVER_FQDN/rudderstack/components/rudder-cli/apply@<VERSION>
 inputs:
 location: "project/"
 access_token: $RUDDERSTACK_ACCESS_TOKEN
 rules:
 - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

Replace <VERSION> with the latest version of the Rudder CLI component.

See the Rudder CLI GitLab component releases for the latest version.

Components

The following Rudder CLI CI/CD components are available for managing transformations:

`validate`

This component validates the project structure and YAML configurations in the specified directory. By default, jobs defined in this component run in the validate stage.

Example

include:
 - component: $CI_SERVER_FQDN/rudderstack/components/rudder-cli/validate@<VERSION>
 inputs:
 location: "project/"
 access_token: $RUDDERSTACK_ACCESS_TOKEN

Inputs

Input	Description	Default value
`location` Required	Path to the folder containing Rudder CLI project files	-
`access_token` Required	RudderStack access token	-
`cli_version`	Version of the Rudder CLI tool to use	`0.13.1`
`stage`	Stage in which the job should run	`validate`

`transformations-test`

This component tests transformations defined in your project by executing their code against test input events. By default, jobs defined in this component run in the test stage.

Example

include:
 - component: $CI_SERVER_FQDN/rudderstack/components/rudder-cli/transformations-test@<VERSION>
 inputs:
 location: "project/"
 access_token: $RUDDERSTACK_ACCESS_TOKEN
 applies_to: "modified"

Inputs

Input	Description	Default value
`location` Required	Path to the folder containing Rudder CLI project files	-
`access_token` Required	RudderStack access token	-
`applies_to`	Test mode: `all` or `modified`	`modified`
`verbose`	Show detailed output including diffs for failures	`false`
`cli_version`	Version of the Rudder CLI tool to use	`0.13.1`
`stage`	Stage in which the job should run	`test`

`apply`

This component syncs the transformation and library definitions in the specified directory to your RudderStack workspace. By default, jobs defined in this component run in the deploy stage.

Example

include:
 - component: $CI_SERVER_FQDN/rudderstack/components/rudder-cli/apply@<VERSION>
 inputs:
 location: "project/"
 access_token: $RUDDERSTACK_ACCESS_TOKEN

Inputs

Input	Description	Default value
`location` Required	Path to the folder containing Rudder CLI project files.	-
`access_token` Required	RudderStack access token	-
`dry_run`	Enable dry-run mode to see changes without applying.	`false`
`cli_version`	Version of the Rudder CLI tool to use.	`0.13.1`
`stage`	Stage in which the job should run.	`deploy`

`transformations-show-default-events`

This component displays the default test events used when no test files are configured for transformations.

Example

include:
 - component: $CI_SERVER_FQDN/rudderstack/components/rudder-cli/transformations-show-default-events@<VERSION>

Inputs

Input	Description	Default value
`cli_version`	Version of the Rudder CLI tool to use.	`0.13.1`
`stage`	Stage in which the job should run.	`test`

How it works

Validation: The validate component ensures your project files are correctly structured before any further steps.
Testing: The transformations-test component runs your transformation code against defined test cases. Using applies_to: modified in merge requests is recommended for faster feedback.
Deployment: When changes are merged into the default branch, the apply component pushes the updates to your RudderStack workspace.

RudderStack Docs – Automate Project Management with Rudder CLI and GitLab CI/CD

Dev-Tools: Automate Transformations Management with Rudder CLI and GitLab CI/CD

Overview

Prerequisites

Token permissions for legacy RBAC system

Setup

Step 1: Configure CI/CD variables

Step 2: Create GitLab CI/CD configuration

Components

validate

Example

Inputs

transformations-test

Example

Inputs

apply

Example

Inputs

transformations-show-default-events

Example

Inputs

How it works

`validate`

`transformations-test`

`apply`

`transformations-show-default-events`