# Reverse ETL Connection YAML Reference

This guide is the YAML reference for Reverse ETL connection resources in a Rudder CLI project.

Reverse ETL connection support is experimental and requires Rudder CLI v0.27.0 or later.

See Enable Reverse ETL connection support for the flags to set before you validate or apply these specs.

Overview

Set kind to retl-connections and version to rudder/v1. spec.connections is a list. Each entry is one Reverse ETL source to destination link, together with its sync configuration. Store the files anywhere under the project root.

Each source must reference a supported Reverse ETL source, and each destination a supported destination. Unlike Event Stream connections, every entry needs a config block.

Each connection uses one of two mapping flows: object mapping if config.object is set (the dashboard’s Visual Data Mapper), JSON mapper otherwise (the JSON Data Mapper). See Mapping flows. The destination spec must set connection_mode.warehouse; see Destination requirements.

Rudder CLI rejects keys it doesn’t recognize at every level of the spec, so a misspelled key fails validation instead of being ignored.

Examples

This connection syncs a SQL model to an HTTP Webhook destination every 30 minutes, sending each row as an identify event:

yaml 
version: rudder/v1
kind: retl-connections
metadata:
  name: retl-connections
spec:
  connections:
    - id: users-to-webhook
      source: "#retl-source-sql-model:users-model"
      destination: "#destination:my-http-destination"
      enabled: true
      config:
        sync_behaviour: upsert
        schedule:
          type: basic
          every_minutes: 30
        identifiers:
          - from: user_id
            to: user_id
        mappings:
          - from: email
            to: traits.email
        event:
          type: identify

Connection fields

Each entry in spec.connections has these fields:

Property Type
Description
id
Required		 String Local identifier. Must be unique across Reverse ETL connection entries in the project. Don’t change it after the first apply.
source
Required		 String Reference to a Reverse ETL source in this project — #retl-source-sql-model:[source.id]. See Supported sources.
destination
Required		 String Reference to a destination in this project — #destination:[destination.id].
enabled Boolean Whether this connection syncs. Defaults to true. Both the source and the destination must also be enabled for the connection to sync; validate warns if they are not.
config
Required		 Object Sync configuration for the connection. See Sync configuration.

Changing source or destination replaces the connection (delete + create). The backend allows only one connection per source–destination pair. See Apply and replace behavior.

Sync configuration

The config object has these fields:

Property Type
Description
sync_behaviour
Required		 String Sync mode — one of upsert, mirror, or full. mirror needs the object mapping flow. A table source on s3 accepts only upsert.

Immutable after the connection is created.
schedule
Required		 Object When the connection syncs. See Schedule.
identifiers
Required		 Array One or more from/to pairs that map a warehouse column to a record identifier. In the JSON mapper flow, every to must be user_id or anonymous_id. In the object mapping flow, supply exactly one entry whose to is the destination field that identifies a record. See Mapping flows.
mappings Array from/to pairs that map a warehouse column to a destination field. Required in the JSON mapper flow.
constants Array key/value pairs sent with every synced record. Both are required and must be non-empty strings, so quote a number or a boolean. JSON mapper flow only.
event Object The event that RudderStack sends for each record. Defaults to identify. JSON mapper flow only. See Event.

Immutable after the connection is created.
object String The destination object to sync to. Setting it selects the object mapping flow. It must be non-empty and carry no leading or trailing whitespace — object: "" fails validation rather than falling back to the JSON mapper flow.

Immutable after the connection is created.
cursor_column String Warehouse column used for incremental syncs. Valid only when sync_behaviour is upsert.

Immutable after the connection is created.
sync_settings Object Sync log and retry settings. See Sync settings.

To change a field marked immutable, see Change an immutable field.

Every from value, cursor_column, and event.name_column is a warehouse column name. It must start with a letter or an underscore, followed by letters, digits, underscores, $, or ..

A constants entry can’t use context.mappedToDestination as its key. RudderStack sets that key itself.

Schedule

The schedule object sets the sync schedule:

Property Type
Description
type
Required		 String One of basic, cron, or manual.
every_minutes Integer Interval between syncs, in minutes. Required when type is basic, and not allowed otherwise. The minimum is 5.
cron_expression String Cron expression for the schedule. Required when type is cron, and not allowed otherwise. See Cron expressions.

A manual schedule takes no other fields. The connection syncs only when you trigger it.

Quote a cron expression. YAML treats a leading * or @ as syntax, so an unquoted */10 * * * * or @hourly fails with a YAML parse error:

yaml 
schedule:
  type: cron
  cron_expression: "*/10 * * * *"

Event

The event object applies to the JSON mapper flow:

Property Type
Description
type
Required		 String One of identify or track.
name String Fixed event name. You can’t set it together with name_column.
name_column String Warehouse column that holds the event name. You can’t set it together with name.

If you omit event, RudderStack sends identify events. Rudder CLI also accepts a track event with neither name nor name_column, but RudderStack has no default event name, so set one. event is immutable, so set it when you create the connection.

This track event reads its name from a column, and a constant adds a fixed value to every record:

yaml 
config:
  event:
    type: track
    name_column: event_name
  constants:
    - key: properties.source
      value: warehouse

See Sync Data from Reverse ETL Sources for how RudderStack sends records as identify and track events.

Sync settings

The sync_settings object controls sync observability. Every field is optional:

Property Type Default
Description
sync_logs.enabled Boolean true Whether RudderStack stores the sync logs in your warehouse.
sync_logs.retention_days Integer 30 Number of days RudderStack keeps the sync logs.
sync_logs.snapshots_to_retain Integer 5 Number of snapshot tables RudderStack keeps.
failed_keys.retry Boolean true Whether RudderStack retries failed records in later syncs.
yaml 
config:
  sync_settings:
    sync_logs:
      enabled: true
      retention_days: 7
      snapshots_to_retain: 3
    failed_keys:
      retry: false

Omitted and removed fields behave as follows:

Mapping flows

A connection maps data in one of two flows. No field names the flow. Rudder CLI derives it from whether object is set.

JSON mapper Object mapping
Selected when object is absent object is set
Destinations Every supported destination Destinations that support object mapping
identifiers One or more, each with to set to user_id or anonymous_id Exactly one, with to set to the destination field that identifies a record
mappings Required Optional
constants Allowed Not allowed
event Allowed Not allowed
mirror sync mode Not available Available

A mappings entry can’t target a field that the flow reserves for identifiers:

Flow Reserved to values
JSON mapper user_id, anonymous_id
Object mapping user_id, context.externalId[0].id

Supported sources

Source Reference Notes
SQL model #retl-source-sql-model:[source.id] Any warehouse type a SQL model supports.
Table source #retl-source-table:[source.id] Experimental. Also set RUDDERSTACK_X_RETL_TABLE_SUPPORT=true (config key flags.retlTableSupport). Without it, Rudder CLI doesn’t recognize the retl-source-table kind. See Rudder CLI Validation Rules for example table source specs.

Reverse ETL sources created from Audiences aren’t supported.

Supported destinations

A Reverse ETL connection can target these destination types. All of them support the JSON mapper flow. In Rudder CLI v0.27.0, you can connect only HTTP Webhook. The others need v0.28.0 or later.

Destination type Object mapping
ActiveCampaign active_campaign No
Amazon S3 s3 No
Amplitude am Yes
Attentive Tag attentive_tag No
BigQuery Stream bqstream No
Braze braze Yes
Facebook Conversions facebook_conversions No
Facebook Pixel facebook_pixel No
Google Analytics 4 ga4 No
Google Cloud Storage gcs No
HTTP Webhook http No
HubSpot hs Yes
Iterable iterable Yes
Mixpanel mp No
PostHog posthog No
TikTok Ads tiktok_ads No
Webhook webhook No

A destination type that doesn’t accept Reverse ETL sources fails validation. The error lists the source types the destination does accept:

destination 'my-googleads-destination' (type 'googleads') does not accept rETL sources: source type 'warehouse' is not among supported source types: web

Customer.io isn’t supported either. It syncs Reverse ETL data through its own flow, which connection specs can’t express.

Destination requirements

The destination’s config must carry a connection_mode entry for the warehouse source type, which is the source type every Reverse ETL source resolves to:

yaml 
config:
  connection_mode:
    warehouse: cloud

Without it, validate reports:

destination 'my-http-destination' config has no 'connection_mode' entry for source type 'warehouse'

Some destination types need additional config keys before a Reverse ETL source can connect:

Destination Required config key
Braze rest_api_key
Facebook Pixel access_token

If the destination already has a Reverse ETL connection that you set up in the dashboard, check its spec after you upgrade to v0.28.0. See Where the mode is declared.

Cron expressions

RudderStack stores a cron_expression as written, without checking it. Rudder CLI checks it locally so that a schedule that can never run, or that runs too often, fails at validate instead of after apply.

Rudder CLI evaluates every expression in UTC. Consecutive syncs must be at least five minutes apart.

Construct Example Result
Five fields: minute, hour, day of month, month, day of week 0 6 * * * Supported
Wildcards, lists, and ranges 0,15 1-5 * * * Supported
Steps on a wildcard or a range */10, 0-30/10 Supported
Steps from a value 15/20 Supported
Month and day names, and name ranges JAN, mon-fri Supported
? in place of *, in any field 0 0 ? * MON Supported — read as *, so a ? in the minute or hour field widens the schedule and can trip the five-minute floor
Lowercase descriptors @hourly, @daily, @midnight, @weekly, @monthly, @yearly, @annually Supported
Descriptor in any other case @DAILY Error
Day of week 7 0 0 * * 7 Error
Wrapping range 22-2 * * * * Error
Wrong field count, value, or step 70 * * * * Error
Six or seven fields (seconds or year) 0 0 0 * * * Error
L, W, and # on the day fields 0 0 L * * Error
Duration shortcuts @every 1h Error
@reboot and unknown descriptors @reboot, @fortnightly Error
Date that never occurs 0 0 30 2 * Error
Syncs less than five minutes apart */2 * * * * Error
CRON_TZ= or TZ= prefix CRON_TZ=UTC 0 6 * * * Warning

An error names the problem:

'cron_expression' is not valid: consecutive syncs have a 2-minute gap; the minimum supported interval is 5 minutes

A timezone prefix is a warning because Rudder CLI can’t check the schedule it produces. The apply goes through, so verify the resulting schedule yourself.

Validation rules

rudder-cli validate runs these checks locally. Error blocks validate and apply. Warning does not.

Check Severity
The spec has every required field, no unrecognized keys, and valid values Error
source and destination use #retl-source-sql-model: (or #retl-source-table:) and #destination: Error
source and destination exist in the project Error
The same source–destination pair appears only once in the project Error
The destination receives from only one Reverse ETL source in the project Error
The destination doesn’t also receive from an Event Stream source in the project Error
The destination type accepts Reverse ETL sources and doesn’t use a destination-specific flow Error
The destination config includes a connection_mode entry for warehouse, and every field its definition requires for that source type Error
object is set only on a destination that supports object mapping Error
identifiers, mappings, constants, and event fit the mapping flow Error
sync_behaviour is supported by the source, the destination, and the mapping flow Error
cursor_column is set only when sync_behaviour is upsert Error
sync_settings is set only for a source that supports it Error
every_minutes is at least 5 Error
cron_expression is valid and syncs no more often than every five minutes Error
cron_expression has no timezone prefix Warning
An enabled connection has both endpoints enabled Warning

Workspace plan gates and permissions aren’t checked locally; apply surfaces the server error. Rudder CLI also doesn’t check that object or a mapped field exists in the destination.

Topology rules see only connections in the project. Remote-only connections (created in the dashboard and never imported) are invisible at validate time.

Import existing connections

bash 
rudder-cli import workspace -l <project-directory>

Enable Reverse ETL connection support before you run the import. Without it, import workspace doesn’t import Reverse ETL connections.

Import writes all importable Reverse ETL connections into one file, imported/retl/connections.yaml. Each connection’s id is generated from the names of its source and destination, for example users-model-to-users-webhook. Related SQL models land under imported/retl/sql-models/. See How to Import Workspace Resources.

Import skips a connection that a spec can’t express and leaves it unchanged in the workspace. Rudder CLI logs each skipped connection, with the reason, to ~/.rudder/cli.log. The most common reasons are:

If the destination is already in your project, its spec needs the connection_mode entry for warehouse before the imported connection passes validate. See Destination requirements.

The first apply after an import can fail with setting external ID for rETL connection during import: …. Run apply again, and the retry adopts the connection.

You can’t import a single Reverse ETL connection by ID. import workspace is the only import path.

See more