Transformation Action

RudderStack’s Transformation Action lets you create, update, test, and publish transformations and libraries written in JavaScript or Python - directly from your development repository. Internally, this action leverages the Transformations API.

This action is currently available for GitHub only.

Prerequisites

• You will need the email address associated with your RudderStack workspace.
• Generate a Service Access Token with Admin privileges.

Note that:

• For production use cases, RudderStack recommends using a Service Access Token instead of Personal Access Token.
• For security purposes, it is highly recommended to use GitHub secrets to store your Service Access Token.

Usage

See the sample workflow for more information on using the action.

name: RudderStack Transformer Test and Publish
uses: rudderlabs/rudder-transformation-action@<current_action_version>
with:
    metaPath: './code/meta.json'
    email: 'test@rudderlabs.com'
    accessToken: ${{ secrets.ACCESS_TOKEN }}
    uploadTestArtifact: true

Specify the email address and Service Access Token obtained in the Prerequisites section in the email and accessToken fields respectively.

See FAQ for more information on the uploadTestArtifact parameter.

Action fields

Meta file structure

As mentioned above, a meta file lets the transformation action know what transformations or libraries to test depending on the input events and the expected output.

A generic meta file schema is shown below:

// Meta file schema
{
  "transformations": <array of transformation schema>,
  "libraries": <array of library schema>
}

A sample meta file is shown below:

 // example meta.json
 {
   "transformations": [
    {
       "file": "./code/t1.js",
       "name": "action-T1",
       "language": "javascript",
       "description": "action-T1",
       "test-input-file": "./code/testevents.json",
       "expected-output": "./code/expectedoutput.json"
     },
     {
       "file": "./code/t2.py",
       "name": "action-T2",
       "language": "pythonfaas",
       "description": "action-T2",
       "test-input-file": "./code/events.json",
       "expected-output": "./code/expected.json"
     }
   ],
   "libraries": [{
       "file": "./code/lib1.js",
       "name": "lib1",
       "language": "javascript",
       "description": "action-lib1"
     },
     {
       "file": "./code/lib2.py",
       "name": "getFinanceDataPy",
       "language": "pythonfaas",
       "description": "Python library to get finance data"
     }
   ]
 }

The path mentioned in the file field should be relative to the base repository path.

Transformation schema

The transformations parameter in the meta file contains an array of transformation schemas. A single transformation schema contains the following parameters:

Library schema

The libraries parameter in the meta file contains an array of library schemas. A single library schema contains the following parameters:

FAQ

How do I use the uploadTestArtifact parameter?

When you set uploadTestArtifact to true in the transformation action, RudderStack:

1. Runs the transformation on the test events present in test-input-file.
2. Generates and stores the output in a test-outputs folder contained within a zipped artifact.
3. Validates the output against the contents of the file specified in expected-output.

Note that:

• RudderStack stores the output from Step 1 in a [transformation name in camel case]_output.json file.
• It stores the validation differences from Step 3 in the respective [transformation name in camel case]_diff.json file.
• If you do not set uploadTestArtifact to true in the transformation action, RudderStack does not create the output zip containing the above files.

For more context, see the use case below.

Consider the following transformation schema:

// single transformationSchema
{
  "file": "./code/t1.js",
  "name": "action-T1",
  "language": "javascript",
  "description": "action-T1",
  "test-input-file": "./code/testevents.json",
  "expected-output": "./code/expectedoutput.json"
}

The workflow when you set uploadTestArtifact to true is as follows:

1. RudderStack runs the transformation action-T1(with its code present in t1.js) on the testevents.json file.

2. RudderStack then:

   1. Generates and stores the output from Step 1 in a file named actionT1_output.json. This file is stored in a test-outputs folder contained within a zipped artifact.
   2. Validates the output with the contents present in the expectedoutput.json file. It stores any differences in a file named actionT1_diff.json.

The test-outputs folder contains the two files - actionT1_output.json and actionT1_diff.json.

Questions? Contact us by email or on Slack