Automate Transformations Management with Rudder CLI and GitLab CI/CD Beta

This guide explains how to validate, test, and manage your RudderStack transformations directly via GitLab CI/CD pipelines using the Rudder CLI CI/CD components.

Overview

By leveraging the Rudder CLI GitLab CI/CD components, you can:

• Validate: Check your transformation YAML configurations for syntax and structure.
• Test: Execute transformation code against test events using the specialized transformations-test component.
• Apply: Deploy your transformations and libraries to your RudderStack workspace automatically.

Prerequisites

• A GitLab repository containing your Rudder CLI project files (transformations and libraries).
• A workspace-level Service Access Token with the following permissions:

• If you’re on Free or self-hosted plan, or for testing and development only: Generate a Personal Access Token with Read-Write role

Any action authenticated by a Personal Access Token will break if the user generating the token is removed from the organization or there is a breaking change to their permissions.

Token permissions for legacy RBAC system

If you are on the legacy Permissions Management (RBAC) system, your workspace-level Service Access Token should have the Admin role and Grant edit access toggled on under Transformations.

See Generate a workspace-level Service Access Token for steps to create the token.

Setup

Follow these steps to set up the GitLab CI/CD pipeline for your transformations.

Step 1: Configure CI/CD variables

1. In your GitLab project, go to Settings > CI/CD > Variables.

2. Add a new variable:

   • Key: RUDDERSTACK_ACCESS_TOKEN
   • Value: The access token generated in the Prerequisites section.
   • Type: Variable
   • Protect variable: Recommended for production branches
   • Mask variable: Recommended to hide the token in logs

Step 2: Create GitLab CI/CD configuration

Create or update your .gitlab-ci.yml file in the root of your repository:

stages: [validate, test, deploy]

include:
  # Validate transformation configurations
  - component: $CI_SERVER_FQDN/rudderstack/components/rudder-cli/validate@<VERSION>
    inputs:
      location: "project/"
      access_token: $RUDDERSTACK_ACCESS_TOKEN

  # Test modified transformations in merge requests
  - component: $CI_SERVER_FQDN/rudderstack/components/rudder-cli/transformations-test@<VERSION>
    inputs:
      location: "project/"
      access_token: $RUDDERSTACK_ACCESS_TOKEN
      applies_to: "modified"
      verbose: true
    rules:
      - if: $CI_PIPELINE_SOURCE == "merge_request_event"

  # Apply changes when merged to the default branch
  - component: $CI_SERVER_FQDN/rudderstack/components/rudder-cli/apply@<VERSION>
    inputs:
      location: "project/"
      access_token: $RUDDERSTACK_ACCESS_TOKEN
    rules:
      - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

Replace <VERSION> with the latest version of the Rudder CLI component.

See the Rudder CLI GitLab component releases for the latest version.

Components

The following Rudder CLI CI/CD components are available for managing transformations:

validate

This component validates the project structure and YAML configurations in the specified directory. By default, jobs defined in this component run in the validate stage.

Example

include:
  - component: $CI_SERVER_FQDN/rudderstack/components/rudder-cli/validate@<VERSION>
    inputs:
      location: "project/"
      access_token: $RUDDERSTACK_ACCESS_TOKEN

Inputs

transformations-test

This component tests transformations defined in your project by executing their code against test input events. By default, jobs defined in this component run in the test stage.

Example

include:
  - component: $CI_SERVER_FQDN/rudderstack/components/rudder-cli/transformations-test@<VERSION>
    inputs:
      location: "project/"
      access_token: $RUDDERSTACK_ACCESS_TOKEN
      applies_to: "modified"

Inputs

apply

This component syncs the transformation and library definitions in the specified directory to your RudderStack workspace. By default, jobs defined in this component run in the deploy stage.

Example

include:
  - component: $CI_SERVER_FQDN/rudderstack/components/rudder-cli/apply@<VERSION>
    inputs:
      location: "project/"
      access_token: $RUDDERSTACK_ACCESS_TOKEN

Inputs

transformations-show-default-events

This component displays the default test events used when no test files are configured for transformations.

Example

include:
  - component: $CI_SERVER_FQDN/rudderstack/components/rudder-cli/transformations-show-default-events@<VERSION>

Inputs

How it works

• Validation: The validate component ensures your project files are correctly structured before any further steps.
• Testing: The transformations-test component runs your transformation code against defined test cases. Using applies_to: modified in merge requests is recommended for faster feedback.
• Deployment: When changes are merged into the default branch, the apply component pushes the updates to your RudderStack workspace.