Tracking Plans Beta

Tracking plans let you proactively monitor and act on non-compliant event data coming into your RudderStack sources based on predefined plans. This can help you prevent or de-risk situations where any missing or improperly configured event data can break your downstream destinations.

Features

• Set rules for track and non-track events (identify, page, screen, and group) that you want to send to downstream destinations. You can use these rules to:

  • Define which events should pass through, including events with and without properties.
  • Specify whether a property or attribute is required and assign the data type required to pass that event.

• Use the Event Audit API to evaluate your inbound events and metadata and compare them with your tracking plans.

• You can iteratively improve your tracking plans and have better control of your data with a robust versioning system.

• Create and manage your tracking plans in the RudderStack dashboard or programmatically using the Data Catalog API.

Create tracking plan

RudderStack provides you with the following options to create tracking plans:

Pull from source

You can create a tracking plan from an existing event data source. This option leverages the Event Audit API to import the events and properties tracked by the event data source and generates an initial plan.

Before you proceed, make sure the Event Audit API setting is turned on in your RudderStack dashboard.

Go to Settings > Workspace and click the Data Management tab. Scroll down to the Data governance section and turn on the Event Audit API toggle.

See Enable Event Audit API section for more information.

1. Log in to the RudderStack dashboard and go to Collect > Tracking Plans option in the left sidebar.
2. Click Create tracking plan.

3. Select Pull from source on the next screen.

4. Select the event stream source from which you want to import the tracked events and properties and click Continue.
5. Enter a unique name and description for your tracking plan and click Continue.
6. Select events tracked from the source and click Continue.

Note that:

• The events and properties listed here are obtained from the Event Audit API.
• RudderStack automatically adds these events and properties to the data catalog if they do not exist already.

7. In the Map properties to events section, RudderStack displays the list of tracked events selected in the above step with the associated properties.

RudderStack supports defining complex nested properties for an event in your tracking plan. Based on the schema sampled from the incoming events, RudderStack shows up to three levels of nesting for a property of object or array data type.

See Nested properties for tracking plans created from source for more information.

8. You can add more properties from the data catalog by going to the Add properties tab and clicking the Add button next to the property you want to add. To mark the property as optional or required, click the Optional/Required option.

In addition, you can:

• Remove an event from the tracking plan.
• Configure the event settings to allow unplanned properties.
• Add properties from the data catalog.
• Remove specific properties from the event.

9. Select the sources you want to connect to the tracking plan and click Continue.

You can connect multiple sources to a tracking plan. However, you can connect a source to only one tracking plan at a given point.

10. Configure the tracking plan settings for specific event types and click Create Tracking Plan.

11. (Optional) Use RudderTyper for autocomplete and linting.

Use tracking plan template

Use this option to import your event and property mappings from a default RudderStack template:

1. Log in to the RudderStack dashboard and go to Collect > Tracking Plans option in the left sidebar.
2. Click Create tracking plan.

3. Click Use template.

4. Select the default RudderStack template and click Continue.

5. Add a tracking plan name and description and click Continue.

6. Map the properties to your events for creating the tracking plan. In addition, you can:

   • Remove an event from the tracking plan.
   • Configure the event settings to allow unplanned properties.
   • Add properties from the data catalog.
   • Remove specific properties from the event.

RudderStack supports defining complex nested properties for an event in your tracking plan. You can add up to three levels of nesting for a property of object or array data type.

7. Connect the tracking plan to the sources.

You can connect multiple sources to a tracking plan. However, you can connect a source to only one tracking plan at a given point.

8. Configure the tracking plan settings for specific event types and click Create Tracking Plan.

From data catalog

You can create a tracking plan from scratch using the events and properties defined in the Data Catalog section:

1. Log in to the RudderStack dashboard and go to Collect > Tracking Plans option in the left sidebar.
2. Click Create tracking plan.

3. Select From data catalog on the next screen.

4. Enter a unique name and description for your tracking plan.
5. Choose the required events from the displayed events list (populated from Data Catalog) and click Continue.

6. In the Map properties to events section, add properties (populated from Data Catalog) to map to each event by clicking the Add properties button.

RudderStack supports defining complex nested properties for an event in your tracking plan. You can add up to three levels of nesting for a property of object or array data type.

7. Click the Add button next to the property you want to add. To mark the property as optional or required, click the Optional/Required option.

In addition, you can:

• Remove an event from the tracking plan.
• Configure the event settings to allow unplanned properties.
• Add properties from the data catalog.
• Remove specific properties from the event.

8. Select the sources you want to connect to the tracking plan and click Continue.

You can connect multiple sources to a tracking plan. However, you can connect a source to only one tracking plan at a given point.

9. Configure the tracking plan settings for specific event types and click Create Tracking Plan.

10. (Optional) Use RudderTyper for autocomplete and linting.

Data Catalog API

You can also use the Data Catalog API to create and manage your tracking plans programmatically.

View tracking plans

To see all the tracking plans associated with your workspace, go to Collect > Tracking Plans. In this view, you get all the tracking plan-related details like name, description, last modified, tracking plan creation method, connected sources, etc.

The following table lists the different ways of creating a tracking plan:

Tracking plan details

Click a tracking plan to view the following details:

• Overview: Displays the list of events and properties associated with the tracking plan. You can edit these by clicking the Edit button.

• Sources: Displays the source(s) connected to the tracking plan. You can view the source details, edit source settings, and disconnect the source from tracking plan by clicking the meatballs menu next to the source.
• Settings: Lets you edit the name of the tracking plan or delete the tracking plan (only if not connected to any source).
• Activity: Lets you view all the activities performed on the tracking plan like events/properties added, removed, or updated etc. along with the user who performed that action.

Migrate tracking plans

RudderStack lets you easily migrate your tracking plans created using the Tracking Plan Spreadsheet to the new format where you can edit events, properties, and tracking plan rules in the dashboard. You will see the Migration now available banner below these tracking plans:

Go to the tracking plan and click Migrate to start the migration process.

Note that:

• You must have Admin privileges to migrate tracking plans successfully.
• Make sure to see the Migration considerations section before you migrate your old tracking plans.

Once the tracking plan is migrated successfully, you can view all the events and properties in the dashboard, and make changes as required.

Considerations

• Once you migrate, note that:

  • RudderStack deletes the old tracking plan.
  • You cannot use the spreadsheet to view or manage the newly created tracking plan.
  • You can manage your tracking plans via the dashboard or the Data Catalog API.

• You will not see the historical violations or event counts once you migrate your tracking plan to the new format in the dashboard.

• RudderStack supports up to three levels of nesting in the event properties of object or array data type. The tracking plan cannot be migrated if one or more of your event mappings have more than three levels of nesting and you will see the following error:

• RudderStack does not permit certain keywords in the event’s JSON schema. You will get an “Unsupported rules found for event” error if you try to migrate tracking plans having these keywords. See the JSON schema guide for the list of all supported keywords.

Edit tracking plans

Go to Collect > Tracking Plans to see all the tracking plans in your workspace. To edit a tracking plan, select a tracking plan and click Edit.

Here, you can:

• Add or remove events and properties from the tracking plan.
• Mark properties as required or optional.
• Define how the information is captured in your events.
• Specify if the tracking plan should allow unplanned properties for a specific event.
• Add advanced rules for an event property.

You cannot change an event type. To do so, delete the event from the data catalog and create a new event.

Click Save to update the tracking plan configuration.

Add advanced rules

While editing a tracking plan, you can also add advanced rules that define how a property should be captured. To do so, click the meatballs (...) menu next to the property and select Add advanced rules. Note that you can set advanced rules only for specific property types.

A panel pops up on the right that lets you define the following rules for the property:

• Enum: Acceptable values for the property.
• Format: Acceptable format in which the property values should be captured. You can choose the required format from the dropdown.
• Pattern: Define the constraints for the property via regular expressions. You can choose the required pattern from the dropdown or define a custom pattern.
• Minimum/maximum length: Acceptable string length for the property.

Event structure for tracking plans

RudderStack applies the tracking plan validation on the following objects by default:

• properties object for track, screen, and page events.
• traits object for identify and group events.

If you are instrumenting your events such that the relevant information is captured in the traits or context.traits object, make sure to select the relevant option from the Identity Applied dropdown.

This dropdown is only available for the identify, page, screen and group events.

Nested event properties

RudderStack supports defining complex nested properties for an event in your tracking plan, allowing you to validate complex data structures in your inbound events and metadata.

To nest event properties while editing your tracking plan:

1. Click Add properties and choose an event property of the object or array data type from the right sidebar. Once added, you should see a + sign next to it.
2. Click the newly added property.
3. Choose the event property to nest.

Note that:

• You can nest properties only within a property of object or array data type.
• RudderStack supports up to three levels of nested properties within an event property.
• If not explicitly declared, RudderStack allows all data types for a property by default. However, it does not support nesting for that property.
• Removing the parent property from the tracking plan automatically removes all the nested properties.
• You cannot nest properties within a property having both array and object data types. An example of such a property is shown:

Use case for tracking plan created from source

For tracking plans created using the Pull from source option, RudderStack populates the events and properties based on the schema sampled from your incoming events. You can define up to three levels of nesting within an event property of object or array data type.

For example, consider the following event payload:

{
  "type": "track",
  "event": "Product Viewed",

  ...

  "properties": {
    "products": [{
        "name": "Product 1",
        "product_id": "a123"
      },
      {
        "name": "Product 2",
        "product_id": "a124"
      }
    ],
    "filters": {
      "value": {
        "price": {
          "max": 10000
        },
        "rating": {
          "max": 5,
          "votes": {
            "min": 1000
          }
        }
      }
    },
  },
  ...
}

Upon importing the above event in your tracking plan, you see the following nested properties within the filters and products properties:

In the above example, RudderStack does not show the property min nested within votes as it exceeds the third level of nesting.

Source-specific settings

Go to the source connected to the tracking plan to see the following settings in the Tracking Plans tab:

• Unlink Tracking Plan: Lets you unlink the tracking plan from your source.

• Previously linked tracking plans: Lets you view the previously linked tracking plans for a source (in the last 30 days).

• Tracking plan settings: Lets you edit the following tracking plan settings for each event type for the connected source. Once done, click Save Settings for the changes to take effect:

  • Drop events with unplanned event names: When toggled on, RudderStack drops all events that do not match the predefined event names in the tracking plan (only applicable for track events).
  • Drop events with unplanned event properties: When turned on, RudderStack drops all events that contain properties not matching the list of predefined properties for the specific event.
  • Drop events with other violations: When toggled on, RudderStack drops all events with violations that include Type Mismatch, Required Fields Missing, and others outlined in the Violation types section.
  • Propagate errors: When turned on, RudderStack captures the validation errors in the event’s context object and sends them downstream (user transformations, destinations), depending on your use-case. If toggled off, RudderStack drops the event containing the validation errors. It is recommended to keep this setting toggled on as it helps you assess the performance of your tracking plans.

Observability

RudderStack gives you complete visibility into the events passing through a tracking plan and details on the tracking plan violations, that is, events that do not comply with the tracking plan rules.

To see these metrics, click the Events tab of the source connected to your tracking plan.

Events ingested

This section shows the tracking plan currently linked to your source and the following details:

• Total events: Captures all events that pass through the source.
• Events validated: Number of events validated by the tracking plan.
• Events with violations: Number of events that do not comply with the tracking plan rules.
• Events dropped: Number of events dropped due to tracking plan violations.

You can see these metrics for all tracking plans connected to that source and filter them by version and time period (past one day, seven days, or 30 days).

Event flow

This section gives a graphical overview of all the events ingested per day over the specified period and the number of events with violations.

Event details

Click the Events tab to see all the event-related metrics like:

• Event name
• Count: Number of events validated by the tracking plan according to filter set in Events ingested.
• Last seen: Number of days since the last observed violation.

Click the Violations tab to see the following metrics:

• Event name
• Event type: Type of event (identify, track, group, page, or screen).

Tracking plans do not validate alias events. If you send any alias events, they are shown in this view and returned as is without validation.

• Events validated: Number of events validated by the tracking plan according to filter set in Events ingested.
• Events with violations: Number of events that did not comply with the tracking plan rules.
• Events dropped: Number of events that were not allowed to flow through.
• Last occurred: Date and time of the last observed violation.

You can also click on an event to view the violation type and individual metrics.

Click the View option for more details, like the violation description and sample payload for the violation.

Violation types

FAQ

Which RudderStack Cloud plans support the tracking plans feature?

The tracking plans feature is supported in the RudderStack Cloud Growth and Enterprise plans.

Although you can create unlimited tracking plans, you can have only five source-tracking plan connections in the RudderStack Cloud Growth plan.

To remove this restriction, upgrade to the Enterprise plan.

Which calls are supported by the tracking plans?

The tracking plans support identify, track, page, screen, and group events.

Note that the alias call is not supported.

Does RudderStack propagate the context related to any tracking plan violations?

RudderStack propagates any context related to the tracking plan violations to your destinations. You can use this context in your transformations for filtering or modifying the events before they reach the destination.

Which keywords are not supported while migrating the tracking plan to the new format in the dashboard?

While migrating your tracking plans to the new format, you will encounter an error if your tracking plan rules contain any of the following advanced keywords as event property names:

• oneOf / allOf
• if / then / else
• $def / $ref
• const
• default

See the JSON schema guide for the list of all supported keywords.

Questions? Contact us by email or on Slack