RudderStack Docs – Manage Data Catalog using Rudder CLI

Archive: Manage Events using Rudder CLI

Mon, 01 Jan 0001 00:00:00 +0000

Events are the foundation of your tracking implementation in RudderStack. This guide shows you how to define and manage events in your Data Catalog using YAML configuration files.

Event types

You can define the following event types in your Data Catalog project using Rudder CLI:

Event type	Description
Track	Record user actions and behaviors
Identify	Capture user identification and associated traits
Page	Track web page views
Screen	Monitor mobile app screen views
Group	Associate users with organizations

Define events

Using your preferred text editor, create a YAML file in your Data Catalog project and add the below content:

Basic structure

You can define events in YAML files with the kind: events specification. See Data Catalog YAML Reference for the detailed YAML spec containing event definitions.

version: rudder/v0.1
kind: events
metadata:
 name: myeventgroup
spec:
 events:
 - id: product_viewed
 name: "Product Viewed" // Only applicable for track events
 event_type: track
 description: "Triggered when a user views a product"
 category: "#/categories/event-categories/browsing_category" # Reference to the Browsing category

Each event definition requires:

A unique identifier (id)
The event type (event_type)
A descriptive name (name) (applicable only for track events)
Optional description (description)
Optional event category (category) — see Manage Event Categories using Rudder CLI for more information.

Do not include the name parameter for identify, page, or group events as they will not pass validation and result in an error.

The following snippets highlight the YAML definitions for different event types:

- id: user_identify
 event_type: identify
 description: "Captures user profile information"
 category: "#/categories/event-categories/signup_category" # Reference to the Signup category

spec:
 events:
 - id: checkout_completed
 name: "Checkout Completed"
 event_type: track
 description: "Triggered when a user completes the checkout process"
 category: "#/categories/event-categories/checkout_category" # Reference to the Checkout category

- id: homepage_viewed
 event_type: page
 description: "Tracks homepage views"
 category: "#/categories/event-categories/browsing_category" # Reference to the Browsing category
- id: settings_screen
 event_type: screen
 description: "Tracks settings screen views"
 category: "#/categories/event-categories/miscellaneous_category" # Reference to the Miscellaneous category

- id: org_association
 event_type: group
 description: "Associates users with their organization"
 category: "#/categories/event-categories/association_category" # Reference to the Association category

Event groups

You can group related events in a single YAML file based on business context (like ecommerce or user management) or other logical categories. Define each group with a unique metadata.name and list related events under spec.events.

When defining event groups, ensure that:

Events in the same group share similar validation requirements.

Related events that are often used together in Tracking Plans are grouped together.

Each event group has a clear, specific purpose (for example, ecommerce_events for ecommerce-related tracking).

The following examples show how to organize events into meaningful groups, along with their YAML definitions:

version: rudder/v0.1
kind: events
metadata:
 name: ecommerce_events
spec:
 events:
 - id: product_viewed
 name: "Product Viewed"
 event_type: track
 description: "Triggered when a user views a product"
 category: "#/categories/event-categories/browsing_category"

 - id: add_to_cart
 name: "Add to Cart"
 event_type: track
 description: "Triggered when a user adds a product to cart"

 - id: checkout_started
 name: "Checkout Started"
 event_type: track
 description: "Triggered when a user starts checkout"

version: rudder/v0.1
kind: events
metadata:
 name: user_events
spec:
 events:
 - id: user_registered
 name: "User Registered"
 event_type: track
 description: "Triggered when a new user registers"
 category: "#/categories/event-categories/signup_category"

 - id: user_login
 name: "User Login"
 event_type: track
 description: "Triggered when a user logs in"

 - id: profile_updated
 name: "Profile Updated"
 event_type: track
 description: "Triggered when a user updates their profile"
 category: "#/categories/event-categories/miscellaneous_category"

Best practices

Follow these best practices when defining events and event groups:

Naming conventions
- Use clear, descriptive names
- Follow consistent capitalization
- Avoid special characters
Organization
- Use event groups to group related events together
- Use meaningful names that reflect the group’s purpose
Validation
- Use unique identifiers
- Validate event definitions
- Test before deployment
Avoid resource duplication
- Do not define the same event in multiple event YAML files. Each event should exist in only one file to prevent duplication and potential confusion.

Validate and deploy events

Before deploying your events to the workspace, validate them to ensure they follow the correct structure and meet your requirements.

Validate events

Run the following command to validate your event definitions:

rudder-cli validate -l ~/tutorial-catalog

The command checks your event definitions for:

Required fields and correct structure
Valid event types and property references
Unique identifiers across your catalog
Proper YAML syntax

If validation succeeds, the command returns no output. If it finds any issues, it displays specific error messages to help you fix them.

Deploy events

After validating your events, deploy them to your RudderStack workspace:

Review the changes before deploying:

rudder-cli apply -l ~/tutorial-catalog --dry-run

Deploy the validated events:

rudder-cli apply -l ~/tutorial-catalog

The above command:

Creates new events in your workspace
Updates existing events if you’ve modified them
Reports the status of each operation
Requires confirmation before making changes (unless you use --confirm=false)

See the End-to-end Walkthrough for steps on validating and deploying events along with other Data Catalog resources.

Next steps

Define properties for your events
Create custom types for validation
Build Tracking Plans using your events and properties
Automate changes to Tracking Plans by leveraging CLI-based workflows

Event categories

Event categories help you organize and classify events in your RudderStack Data Catalog based on business context, functionality, or any other logical classification system. This section shows you how to define event categories and associate them with events using YAML configuration files.

Overview

In the RudderStack dashboard, you can use the Data Catalog to:

Define custom categories for your events
Assign predefined categories to your events

You can also use Rudder CLI to define event categories as YAML configuration files in your Data Catalog project and reference them in your event definitions.

Define event categories

Using your preferred text editor, create a YAML file in your Data Catalog project and add the below content:

Basic structure

You can define event categories in YAML files with the kind: categories specification. See Data Catalog YAML Reference for the detailed YAML spec containing event definitions.

version: rudder/v0.1
kind: categories
metadata:
 name: event-categories
spec:
 categories:
 - id: signup_category
 name: Signup
 - id: login_category
 name: Login
 - id: browsing_category
 name: Browsing
 - id: miscellaneous_category
 name: Miscellaneous

Each event category definition requires:

A unique identifier (id)
A descriptive name (name)

Best practices

Follow these best practices when defining event categories:

Naming conventions
- Use clear, descriptive category names that reflect their purpose
- Follow consistent capitalization (for example, Marketing instead of marketing)
- Keep names concise but meaningful
Organization
- Create categories that align with your business objectives
- Consider your team’s workflow and how events will be used
- Avoid creating too many categories that could complicate organization
Uniqueness
- Ensure each category has a unique identifier
- Avoid duplicate category names within the same configuration
- Use descriptive IDs that relate to the category name (for example, marketing_category for “Marketing”)

Reference categories in events

Once you’ve defined your categories, you can associate them with events in your event definitions.

Each event can have only one category assigned to it. See Limitations for the other key points to consider while referencing categories in events.

To reference a category in an event definition, use the category field with a $ref pointer to your category:

version: rudder/v0.1
kind: events
metadata:
 name: categorized-events
spec:
 events:
 - id: product_viewed
 event_type: track
 name: "Product Viewed"
 category: "#/categories/event-categories/browsing_category"

 - id: signup_completed
 event_type: track
 name: "Signup Completed"
 category: "#/categories/event-categories/signup_category"

The $ref format follows the below syntax:

"#/categories/{categories-metadata-name}/{category-id}"

Where:

categories-metadata-name is the name specified in your categories file’s metadata.
category-id is the unique identifier of the specific category.

Validate and deploy event categories

Before deploying your event categories to the workspace, validate them to ensure they follow the correct structure and meet your requirements.

Validate categories

Run the following command to validate your category definitions and event associations:

rudder-cli validate -l ~/tutorial-catalog

The command checks your category definitions for:

Required fields and correct structure
Unique category identifiers and names
Valid category references in event definitions
Proper YAML syntax and formatting
No duplicate categories within the same configuration

If validation succeeds, the command returns no output. If it finds any issues, it displays specific error messages to help you fix them.

Deploy categories

After validating your categories and events, deploy them to your RudderStack workspace:

Review the changes before deploying:

rudder-cli apply -l ~/tutorial-catalog --dry-run

Deploy the validated categories:

rudder-cli apply -l ~/tutorial-catalog

The above command:

Creates new categories in your workspace
Updates existing categories if you’ve modified them
Associates events with their specified categories
Reports the status of each operation
Requires confirmation before making changes (unless you use --confirm=false)

Categories are applied alongside other Data Catalog resources. When you deploy, both your categories and the events that reference them will be updated in your workspace.

See the End-to-end Walkthrough for steps on validating and deploying event categories along with other Data Catalog resources.

Limitations

The current Rudder CLI implementation of event categories has the following limitations:

Event-only support: Categories can only be assigned to events. Other Data Catalog assets like properties or custom types do not support categories in this version.
Single category per event: Each event can be assigned to only one category. Multiple categories per event are not supported.
No duplicate categories: You cannot create duplicate categories with the same name, regardless of whether they are defined within the same configuration file or across different configuration files.

Next steps

Create events and reference the event categories in them
Define properties for your categorized events
Build Tracking Plans using your categorized events
Explore the Data Catalog YAML Reference for advanced configuration options
Automate changes to categories and events by leveraging CLI-based workflows

Archive: Manage Properties using Rudder CLI

Mon, 01 Jan 0001 00:00:00 +0000

Properties provide context to your events by capturing additional attributes and metadata. This guide shows you how to define and manage properties in your Data Catalog using YAML configuration files.

Property types

RudderStack supports the following property types in your Data Catalog:

Property type	Description
String	Text values
Integer/Number	Number values
Boolean	True/False values
Array	Lists of values
Object	Nested structures
Custom Types	User-defined types with specific validation rules

Define properties

Using your preferred text editor, create a YAML file in your Data Catalog project and add the below content:

Basic structure

You can define properties in YAML files with the kind: properties specification. See Data Catalog YAML Reference for the detailed YAML spec containing property definitions.

version: rudder/v0.1
kind: properties
metadata:
 name: mypropertygroup
spec:
 properties:
 - id: user_email
 name: "Email"
 type: string
 description: "User's email address"

Each property definition requires:

A unique identifier (id)
A property name (name)
A data type (type)
Optional description (description) and validation rules

The following snippets highlight the YAML definitions for different property types:

- id: product_price
 name: "Price"
 type: number
 description: "Product price in USD"

- id: is_premium
 name: "isPremium"
 type: boolean
 description: "Whether the user has a premium subscription"

- id: username
 name: "Username"
 type: string
 description: "User's name"
 propConfig:
 minLength: 3
 maxLength: 50
 pattern: "^[a-zA-Z0-9_-]+$"

- id: age
 name: "Age"
 type: integer
 description: "User's age"
 propConfig:
 minimum: 13
 maximum: 120

- id: categories
 name: "Categories"
 type: array
 description: "User's favorite product categories"
 propConfig:
 minItems: 1
 maxItems: 5
 uniqueItems: true

- id: shipping_address
 name: "Address"
 type: object
 description: "User's shipping address"
 properties:
 - id: street
 type: string
 required: true
 - id: city
 type: string
 required: true
 - id: postal_code
 type: string
 required: true
 - id: country
 type: string
 required: true

RudderStack supports nesting properties while defining Tracking Plan rules — this lets you validate hierarchical data structures that mirror your actual data model.

Property groups

You can group related properties in a single YAML file based on business context (like user profiles or product details) or other logical categories. Define each group with a unique metadata.name and list related properties under spec.properties.

When defining property groups, ensure that:

Properties in the same group share similar validation requirements

Related properties that are often used together in Tracking Plans are grouped together

Each property group has a clear, specific purpose (for example, user_properties for user-related attributes)

The following examples show how to organize properties into meaningful groups:

version: rudder/v0.1
kind: properties
metadata:
 name: user_properties
spec:
 properties:
 - id: user_id
 name: "ID"
 type: string
 description: "Unique identifier for the user"
 propConfig:
 minLength: 1
 maxLength: 64

 - id: email
 name: "Email"
 type: string
 description: "User's email address"
 propConfig:
 format: "email"

 - id: account_type
 name: "Account Type"
 type: string
 description: "Type of user account"
 propConfig:
 enum:
 - "personal"
 - "business"
 - "enterprise"

version: rudder/v0.1
kind: properties
metadata:
 name: product_properties
spec:
 properties:
 - id: product_id
 name: "Product ID"
 type: string
 description: "Unique identifier for the product"
 propConfig:
 pattern: "^PRD-[0-9]{6}$"

 - id: price
 name: "Price"
 type: number
 description: "Product price in USD"
 propConfig:
 minimum: 0
 exclusiveMinimum: true

 - id: categories
 name: "Categories"
 type: array
 description: "Product categories"
 propConfig:
 minItems: 1
 maxItems: 3
 uniqueItems: true

Best practices

Follow these best practices when defining properties and property groups:

Naming conventions
- Use clear, descriptive names
- Follow consistent casing
- Use meaningful prefixes for grouping
Organization
- Use property groups to group related properties together
- Use meaningful names that reflect the group’s purpose
Validation
- Add meaningful constraints
- Document validation requirements
- Test edge cases
Avoid resource duplication
- Do not define the same property in multiple property YAML files. Each property should exist in only one file to prevent duplication and potential confusion.

Validate and deploy properties

Before deploying your properties to the workspace, validate them to ensure they follow the correct structure and meet your requirements.

Validate properties

Run the following command to validate your property definitions:

rudder-cli validate -l ~/tutorial-catalog

The command checks your property definitions for:

Required fields and correct structure
Valid property types and validation rules
Unique identifiers across your catalog
Proper YAML syntax

If validation succeeds, the command returns no output. If it finds any issues, it displays specific error messages to help you fix them.

Deploy properties

After validating your properties, deploy them to your RudderStack workspace:

Review the changes before deploying:

rudder-cli apply -l ~/tutorial-catalog --dry-run

Deploy the validated properties:

rudder-cli apply -l ~/tutorial-catalog

The above command:

Creates new properties in your workspace
Updates existing properties if you’ve modified them
Reports the status of each operation
Requires confirmation before making changes (unless you use --confirm=false)

See the End-to-end Walkthrough for steps on validating and deploying properties along with other Data Catalog resources.

Next steps

Create custom types for complex validation
Associate properties with events
Build Tracking Plans using your events and properties
Automate changes to Tracking Plans by leveraging CLI-based workflows

Archive: Use Custom Data Types in Rudder CLI

Mon, 01 Jan 0001 00:00:00 +0000

This guide walks you through creating and managing custom data types as YAML files in Rudder CLI.

Overview

Custom Data Types extend RudderStack’s CLI-based Tracking Plan management capabilities, allowing you to define reusable data validation patterns that can be referenced across multiple properties in your RudderStack Tracking Plans, improving consistency and reducing duplication.

With custom data types, you can:

Define custom data types in YAML files with specific validation rules
Reference these custom data types in property definitions
Validate and sync using existing Rudder CLI commands
Version control your data types along with events, properties, and Tracking Plans
Deploy to your RudderStack workspace through your established Git workflow

Use cases

This section highlights some common scenarios where custom data types can be helpful.

Standardizing validation

Alice is a Product Manager responsible for maintaining data quality across her organization. She needs to ensure that email properties consistently follow the same validation rules regardless of which team implements them.

With custom data types, she creates an EmailType with specific validation rules once, and then teams throughout the organization can reference this type in their property definitions, ensuring consistency.

Simplifying implementation

Bob is a Product Engineer implementing event tracking for a new feature.

Instead of figuring out the correct validation rules for each property, he references custom types defined by Alice. This ensures his implementation follows company standards without requiring detailed knowledge of validation patterns.

Define custom types

You can define custom data types in your Rudder CLI project as YAML files with the required validation rules.

A sample YAML file ~/tutorial-catalog/my-custom-types.yaml that defines two custom data types (SKU Type and Category Type) is shown:

version: rudder/v0.1
kind: custom-types
metadata:
 name: identifier-types
spec:
 types:
 - id: sku_type
 name: "SKUType"
 description: "Custom type for SKU validation"
 type: string
 config: # Validation rules for the custom data type
 minLength: 5
 maxLength: 255
 pattern: "^SKU-[0-9]+$"
 - id: category_type
 name: "CategoryType"
 description: "Custom type for product identifiers"
 type: string
 config: # Validation rules for the custom data type
 minLength: 10
 maxLength: 20
 pattern: "^PROD-[0-9]+$"

You can then reference these custom data types in your property definitions.

YAML spec

You can define custom data types in the YAML file by setting kind to custom-types.

The spec parameter of the YAML file has the following structure:

Property	Type	Description
`types` Required	Array of custom type definitions	An array of custom type objects grouped together in the same file.

Custom type definition

Property	Type	Description
`id` Required	String	Unique identifier for the custom type within the project. This parameter must be unique across all custom types in all the YAML files within the project.
`name` Required	String	Display name of the custom type. Note: The custom type name must be between 2 and 65 characters long. It must start with a capital letter and contain only letters, numbers, underscores and dashes. Spaces are not allowed.
`type` Required	String	Base data type for the custom type. Acceptable values are: `string`, `integer`, `number`, `object`, `array`, `boolean`.
`config` Required	`config` object	Validation rules for the custom type. The configuration options vary depending on the `type` parameter.
`description`	String	Description of the custom type.

`config` options

The config object’s configuration varies depending on the type parameter.

Parameter	Description
`minLength`	Minimum string length
`maxLength`	Maximum string length
`pattern`	Regular expression patterns. RudderStack supports the following patterns: Email Date-time Date Time URL IPv4 IPv6 Custom (define a custom pattern)
`format`	Predefined format in which the values should be captured. RudderStack accepts the following formats: date-time date time email hostname ipv4 ipv6 uuid
`enum`	Array of acceptable values

Parameter	Description
`minimum`	Minimum value
`maximum`	Maximum value
`exclusiveMinimum`	Exclusive minimum value
`exclusiveMaximum`	Exclusive maximum value
`multipleOf`	Multiple of value

Parameter	Description
`itemTypes`	List of acceptable types for array items
`minItems`	Minimum number of items
`maxItems`	Maximum number of items
`uniqueItems`	Boolean requiring uniqueness of items

Best practices

Follow these best practices when defining custom types:

Naming conventions
- Use clear, descriptive names
- Follow consistent capitalization
- Use meaningful prefixes for grouping
Organization
- Group related custom types together in the same file
- Use meaningful names that reflect the type’s purpose
Validation
- Add meaningful constraints
- Document validation requirements
- Test edge cases
Avoid resource duplication
- Do not define the same custom type in multiple custom type YAML files. Each custom type should exist in only one file to prevent duplication and potential confusion.

Reference custom types in properties

Properties can reference custom types using the standard reference format consistent with how events and properties are referenced in Tracking Plans.

The following snippet references two custom types (SKU Type and Category Type) in the properties SKU and Category, respectively:

version: rudder/v0.1
kind: properties
metadata:
 name: mypropertygroup
spec:
 properties:
 - id: product_sku
 name: "SKU"
 type: "#/custom-types/identifier-types/sku_type" # Reference to custom type
 description: "Product SKU"
 - id: category
 name: "Category"
 type: "#/custom-types/identifier-types/category_type" # Reference to custom type
 description: "Product's category"

Note that:

Custom types can reference other custom types, allowing for complex type hierarchies. See Advanced custom type features for more information.

While there’s no limit on nesting depth, circular references are not supported currently.

Advanced features

This section covers some advanced use cases where you can leverage custom types.

Array of custom types

You can define arrays of custom types, as shown:

version: rudder/v0.1
kind: custom-types
metadata:
 name: advanced-types
spec:
 types:
 - id: "emails_array_type"
 name: "EmailArrayType"
 description: "Array of email addresses"
 type: array
 config:
 itemTypes:
 - "#/custom-types/email-types/email_type" # Reference to another custom type
 minItems: 1
 maxItems: 10
 uniqueItems: true

Object custom types

You can define object types with specific properties:

version: rudder/v0.1
kind: custom-types
metadata:
 name: object-types
spec:
 types:
 - id: address_type
 name: "AddressType"
 description: "Physical address information"
 type: object
 properties:
 - $ref: "#/properties/address/street"
 required: true
 - $ref: "#/properties/address/city"
 required: true
 - $ref: "#/properties/address/zip"
 required: true
 - $ref: "#/properties/address/country"
 required: false

Validate and deploy custom types

Before deploying your custom types to the workspace, validate them to ensure they follow the correct structure and meet your requirements.

Validate custom types

Run the following command to validate your custom type definitions:

rudder-cli validate -l ~/tutorial-catalog

The command checks your custom type definitions for:

Required fields and correct structure
Valid custom type definitions
Proper YAML syntax

If validation succeeds, the command returns no output. If it finds any issues, it displays specific error messages to help you fix them.

Deploy custom types

After validating your custom types, deploy them to your RudderStack workspace:

Review the changes before deploying:

rudder-cli apply -l ~/tutorial-catalog --dry-run

Deploy the validated changes:

rudder-cli apply -l ~/tutorial-catalog

The above command:

Creates new custom types in your workspace
Updates existing custom types if you’ve modified them
Reports the status of each operation
Requires confirmation before making changes (unless you use --confirm=false)

See the End-to-end Walkthrough for steps on validating and deploying custom types along with other Data Catalog resources.

Limitations

Circular reference checks are not currently performed.
When modifying a custom type that’s already in use, the changes will propagate to all properties that reference this custom type, potentially affecting validation behavior across all related properties.

FAQ

How do custom types differ from property definitions?

Custom types are reusable type definitions that can be referenced by multiple properties. They encapsulate validation rules that can be consistently applied across multiple properties.

Will new custom data types work with existing Tracking Plans?

Yes, existing Tracking Plans will continue to work without modification. Custom types are an enhancement that you can adopt gradually.

RudderStack Docs – Manage Data Catalog using Rudder CLI

Archive: Manage Events using Rudder CLI

Event types

Define events

Basic structure

Event groups

Best practices

Validate and deploy events

Validate events

Deploy events

Next steps

Event categories

Overview

Define event categories

Basic structure

Best practices

Reference categories in events

Validate and deploy event categories

Validate categories

Deploy categories

Limitations

Next steps

Archive: Manage Properties using Rudder CLI

Property types

Define properties

Basic structure

Property groups

Best practices

Validate and deploy properties

Validate properties

Deploy properties

Next steps

Archive: Use Custom Data Types in Rudder CLI

Overview

Use cases

Standardizing validation

Simplifying implementation

Define custom types

YAML spec

Custom type definition

config options

Best practices

Reference custom types in properties

Advanced features

Array of custom types

Object custom types

Validate and deploy custom types

Validate custom types

Deploy custom types

Limitations

FAQ

How do custom types differ from property definitions?

Will new custom data types work with existing Tracking Plans?

`config` options