Select spec version:

Google Cloud Pub/Sub Destination Config Reference

Beta

Complete Rudder CLI reference for the Google Cloud Pub/Sub destination config keys, topic and attribute mapping, source types, and secrets.

Available Plans

free
growth
enterprise

6 minute read

Google Cloud Pub/Sub is a streaming destination. RudderStack publishes each event as a JSON message to the Pub/Sub topic its name or type maps to, optionally with message attributes copied from the event.

In a Google Cloud Pub/Sub destination spec:

type: googlepubsub
definition_version: 1

Sample configuration

yaml

version: rudder/v1
kind: destination
metadata:
  name: pubsub-prod
spec:
  id: pubsub-prod
  display_name: PubSub Production
  type: googlepubsub
  definition_version: 1
  enabled: true
  config:
    project_id: acme-analytics
    credentials: '{{ .PUBSUB_CREDENTIALS }}'

    event_to_topic_map:
      - from: Order Completed
        to: rudder-orders
      - from: identify
        to: rudder-users
      - from: "*"
        to: rudder-events
    event_to_attribute_map:
      - from: Order Completed
        to: order_id
      - from: Order Completed
        to: currency

    connection_mode:
      web: cloud
      cloud: cloud
    consent_management:
      web:
        - provider: oneTrust
          consents:
            - analytics

The above example sends Order Completed and identify events to their own topics, and every other event to rudder-events — see Topic mapping. credentials is JSON, so its reference is single-quoted — see Secrets.

Config keys

config accepts only the keys listed below. The shared config key rules cover unknown keys, defaults, and immutability.

Connection

project_id

Required

Type: string

Description: ID of the GCP project that holds the topics.

Notes:

At most 100 characters, and must not contain line breaks.
A {{ path || fallback }} template is accepted by validate, but RudderStack connects with the project ID as written, so use a literal.

credentials

RequiredSecret

Type: string

Description: Contents of the JSON key file for a GCP service account with the Pub/Sub Publisher role on the topics.

Notes:

Only checked for presence locally. RudderStack accepts only a service account key — any other kind of Google credential fails at delivery.

Supply it as a {{ .VAR }} reference rather than a literal — see Secrets.

See Get started for creating the service account and its key.

Topic mapping

Each event is matched against the from field of both mappings by its event name first, then by its type — track, page, identify, and so on — then by *, which matches every event. Matching ignores case.

event_to_topic_map

Type: array of objects

Description: Maps events to the topics RudderStack publishes them to.

Notes:

from — an event name, an event type, or *.
to — ID of the topic, exactly as it appears in Pub/Sub. Topic IDs are case-sensitive.
Each field is at most 100 characters, and must not contain line breaks.
A {{ path || fallback }} template is accepted by validate in either field, but RudderStack prepares a publisher for each to value as written, so keep to literal.

Rudder CLI doesn’t require this key, but an event that matches no entry isn’t delivered. Map * to send everything else to one topic:

yaml

event_to_topic_map:
  - from: Order Completed
    to: rudder-orders
  - from: "*"
    to: rudder-events

Quote "*" — unquoted, YAML reads it as an alias and the spec fails to parse.

See How does event mapping work with the topic ID?.

event_to_attribute_map

Type: array of objects

Description: Adds message attributes copied from the event. Each entry names a field whose value RudderStack attaches as an attribute.

Notes:

from — an event name, an event type, or *, matched as described above. Only one set of entries applies to an event: those for its name, or else its type, or else *.
to — path of the field, in dot notation. RudderStack looks it up at the event’s root, then in properties, traits, and context.traits. The attribute takes the last segment of the path as its name, so metadata.order_id becomes order_id.
Repeat a from to attach several attributes. A field the event doesn’t carry adds no attribute.
Attribute values are strings. A number or boolean is converted, and an object or array is sent as JSON.
Each field is at most 100 characters, and must not contain line breaks. Both accept a {{ path || fallback }} template in place of a literal.

yaml

event_to_attribute_map:
  - from: Order Completed
    to: order_id
  - from: Order Completed
    to: metadata.coupon

See Attributes.

Per-source keys

Both keys are objects keyed by the local source type — the tokens listed under Source types. A key naming a source type this destination doesn’t support fails validation.

connection_mode

Type: object

Description: Maps each source type you connect to the mode its events reach Pub/Sub in, using the modes in Source types.

Notes:

An entry is required for each source type you connect — see Connect a source.

yaml

connection_mode:
  web: cloud
  cloud: cloud

consent_management

Type: object

Description: Consent provider configuration per source type. The entry shape, accepted providers, and the rules on resolution_strategy and consents are shared across all destination types — see Consent management.

Source types

Google Cloud Pub/Sub accepts events from these source types in the mentioned connection modes:

Source type	Connection mode
`android`	`cloud`
`android_kotlin`	`cloud`
`ios`	`cloud`
`ios_swift`	`cloud`
`web`	`cloud`
`unity`	`cloud`
`react_native`	`cloud`
`flutter`	`cloud`
`cordova`	`cloud`
`cloud`	`cloud`
`warehouse`	`cloud`

Every source type is cloud only — events reach the topics from RudderStack’s servers, never in device mode.

warehouse is the token a Reverse ETL source resolves to. Rudder CLI manages the destination’s warehouse entry — if the destination is connected to a Reverse ETL source, set connection_mode.warehouse: cloud in its spec, or apply removes it.

Rudder CLI can also declare the Reverse ETL connection itself, for the types listed under Supported destinations — see Manage Reverse ETL Connections using Rudder CLI.

The dashboard additionally offers Google Cloud Pub/Sub to AMP and Shopify sources. Rudder CLI doesn’t manage those connections, so amp and shopify are invalid here.

Connect a source

An event stream connection to this destination is checked against two rules at validate time.

The source’s type must be supported. A source’s type resolves to one of the tokens above before the check — a JavaScript source resolves to web, and webhook and server-side SDK sources resolve to cloud. Every type an event stream source can resolve to is supported here, so this check always passes.

The destination config must carry a connection_mode entry for that source type. This lives on the destination spec, not on the connection spec. Without it:

destination 'pubsub-prod' config has no 'connection_mode' entry for source type 'web'

Google Cloud Pub/Sub needs no additional config keys to connect a source of any type.

Secrets

credentials is the only secret key. Write it as a {{ .VAR }} reference and supply the value at apply time:

yaml

config:
  credentials: '{{ .PUBSUB_CREDENTIALS }}'

bash

export RUDDER_PUBSUB_CREDENTIALS="$(cat service-account.json)"
rudder-cli apply

# or
rudder-cli apply --var-file secrets.vars.yaml

Note that:

credentials is JSON, so write its reference in single quotes, as above. Rudder CLI pastes the value into the YAML before parsing it, and the JSON’s double quotes would end a double-quoted string.
rudder-cli import writes only the secrets the API returns, as {{ .VAR }} references, and leaves the others out. Add any secret key it leaves out, then supply the variables before you apply — see Secrets.

See How to Use Variable Substitution in Rudder CLI.

Changes by release

No changes since this type became available.

For changes across every destination type, see Destination Type Changes by Release.

See more

Google Pub/Sub Destination for service account setup, topic mapping, and attributes
Destination Type Reference for Rudder CLI for the rules shared across destination types
Destination YAML Reference for the spec envelope

Google Cloud Pub/Sub Destination Config Reference

Sample configuration

Config keys

Connection

Topic mapping

Per-source keys

Source types

Connect a source

Secrets

Changes by release

See more

Questions? Let's figure it out together.