# Google BigQuery Destination Config Reference

Google BigQuery is a warehouse destination. RudderStack stages events as files in a Google Cloud Storage bucket, then loads them into a BigQuery dataset on a schedule.

In a BigQuery destination spec:

Sample configuration

yaml 
version: rudder/v1
kind: destination
metadata:
  name: bigquery-prod
spec:
  id: bigquery-prod
  display_name: BigQuery Production
  type: bq
  definition_version: 1
  enabled: true
  config:
    project: acme-analytics
    location: US
    bucket_name: rudder-bq-staging
    prefix: rudder/events
    namespace: rudder_events
    credentials: "{{ .BQ_CREDENTIALS }}"

    sync_frequency: "180"
    sync_start_at: "01:00"
    exclude_window:
      start_time: "02:00"
      end_time: "03:00"

    skip_users_table: true
    skip_tracks_table: false
    skip_views: false
    partition_column: loaded_at
    partition_type: day
    json_paths: context.traits,properties.metadata
    cleanup_object_storage_files: false

    underscore_divide_numbers: false
    allow_users_context_traits: false

    connection_mode:
      web: cloud
      android_kotlin: cloud
    consent_management:
      web:
        - provider: oneTrust
          consents:
            - analytics
            - marketing

The above example uses a {{ .VAR }} reference for credentials — see Secrets.

Config keys

config accepts only the keys listed below. The shared config key rules cover unknown keys, defaults, and immutability.

BigQuery’s string keys don’t accept {{ path || fallback }} templates as a way around their constraints. A template is measured as literal text against the same rule, so an over-long one fails, and a key constrained by shape rather than length — bucket_name, partition_column, partition_type — rejects a template outright. Use {{ .VAR }} substitution for credentials, which has no pattern constraint.

Connection

project

Required

Type: string

Description: GCP project ID that holds the BigQuery dataset.

Notes:

  • At most 100 characters, and must not contain line breaks.

location

Type: string

Description: GCP region the dataset lives in, for example US, EU, or asia-southeast1.

Notes:

  • At most 100 characters, and must not contain line breaks.

bucket_name

Required

Type: string

Description: Staging GCS bucket RudderStack writes event files to before loading them into BigQuery. The bucket must already exist, and should be co-located with the dataset so loads don’t cross regions.

Notes:

  • 3 to 63 characters, matching [a-z0-9][a-z0-9-._]{1,61}[a-z0-9].
  • It must not start with goog, contain google, look like an IP address, or contain consecutive dots.

prefix

Type: string

Description: Folder prefix inside the staging bucket. RudderStack creates a folder with this prefix and writes all staged files beneath it.

Notes:

  • At most 100 characters, and must not contain line breaks.

namespace

Immutable

Type: string

Description: Dataset RudderStack creates its tables in. Defaults to the source name, snake-cased, when omitted.

Notes:

  • At most 64 characters, and must not start with pg_ in any capitalization.
  • Can’t be changed once the destination exists — the API rejects the update. Create a new destination instead.

credentials

RequiredSecret

Type: string

Description: GCP service account JSON key. The account needs BigQuery dataset, table, and job permissions, plus read and write access to the staging bucket.

Supply it as a {{ .VAR }} reference rather than a literal — see Secrets.

See Setting up the service account for RudderStack for the exact roles.

Sync scheduling

sync_frequency

Required

Type: string

Description: How often RudderStack syncs staged events into the dataset, in minutes. Written as a string, not a number.

Notes:

  • One of 5, 10, 15, 30, 60, 180, 360, 720, or 1440.
  • The dashboard defaults this field to 180. Rudder CLI requires it explicitly.
  • A spec that omits this key fails validation.

sync_start_at

Type: string

Description: Time of day, in UTC, that anchors the sync schedule. Subsequent syncs are computed from it at sync_frequency intervals. Written as HH:MM — the dashboard offers 15-minute steps.

Notes:

  • Not validated locally: any string is accepted, and a value the warehouse scheduler can’t parse silently yields no scheduled times.

exclude_window

Type: object

Description: Daily window, in UTC, during which RudderStack doesn’t sync. Omit the block entirely to sync around the clock.

Notes:

  • When present, both fields are required.
  • start_time — string, when the window opens, HH:MM
  • end_time — string, when the window closes, HH:MM
  • Neither field’s format is validated locally.
yaml 
exclude_window:
  start_time: "02:00"
  end_time: "03:00"

Table behavior

skip_users_table

Type: boolean

Default value: true

Description: Send identify events only to the identifies table, skipping the users table. The users table holds one row per unique user and is maintained with a merge, which can add significant time to each sync.

Set it to false to populate both tables. See Warehouse Schema.

skip_tracks_table

Type: boolean

Default value: false

Description: Skip sending events to the tracks table. Per-event tables are unaffected.

skip_views

Immutable

Type: boolean

Default value: false

Description: Skip creating the <table_name>_view deduplication view alongside each table. The views cover the last 60 days and exist so queries don’t return duplicate events. Skip them only if you deduplicate another way.

Notes:

  • Can’t be changed once the destination exists — the API rejects the update.
See Skip views creation.

partition_column

Immutable

Type: string

Default value: _PARTITIONTIME

Description: Column BigQuery partitions each table on:

Notes:

  • _PARTITIONTIME — ingestion time, when BigQuery received the data
  • loaded_at — when RudderStack loaded the data into the warehouse
  • received_at — when RudderStack received the event
  • timestamp — event time corrected for client-side clock skew
  • sent_at — when the client sent the event to RudderStack
  • original_timestamp — when the event was generated at the source
  • Can’t be changed once the destination exists — the API rejects the update. Create a new destination instead.
For what each timestamp means, see Common Fields. For how partitioning works here, see Partitioned tables.

partition_type

Immutable

Type: string

Default value: day

Description: Granularity of the partition: hour or day.

Notes:

  • Can’t be changed once the destination exists, on the same terms as partition_column.

json_paths

Type: string

Description: Comma-separated dot-notation paths whose values are stored as JSON columns instead of being flattened into separate columns. Applies to every track event sent to this destination.

Notes:

  • Not validated locally.
yaml 
json_paths: context.traits,properties.metadata

See JSON Column Support.

cleanup_object_storage_files

Type: boolean

Default value: false

Description: Delete staged files from the GCS bucket after a sync completes successfully.

Legacy column naming

Both keys below exist to preserve the column naming of destinations created before the behavior changed. Leave them at their defaults on a new destination. Neither can be changed once the destination exists — the API rejects the update.

underscore_divide_numbers

Immutable

Type: boolean

Default value: false

Description: When false, numeric suffixes in column names are preserved: v3 stays v3 rather than being split into v_3.

allow_users_context_traits

Immutable

Type: boolean

Default value: false

Description: When false, context.traits.* fields aren’t promoted to top-level traits and are stored only as context_traits_* columns.

Per-source keys

Both keys are objects keyed by the local source type — the tokens listed under Source types. A key naming a source type this destination doesn’t support fails validation.

connection_mode

Type: object

Description: Maps each source type you connect to the mode its events reach BigQuery in, using the modes in Source types.

Notes:

  • An entry is required for each source type you connect — see Connect a source.
yaml 
connection_mode:
  web: cloud
  android_kotlin: cloud

Source types

Google BigQuery accepts events from these source types in the mentioned connection modes:

Source type Connection mode
android cloud
android_kotlin cloud
ios cloud
ios_swift cloud
web cloud
unity cloud
cloud cloud
react_native cloud
flutter cloud
cordova cloud

Every source type is cloud only — events reach the dataset from RudderStack’s servers, never in device mode.

The dashboard additionally offers BigQuery to AMP, Shopify, and cloud app sources. Rudder CLI doesn’t manage those connections, so amp, shopify, and cloud_source are invalid here. warehouse isn’t valid either — BigQuery doesn’t accept it even in the dashboard.

Connect a source

An event stream connection to this destination is checked against two rules at validate time.

The source’s type must be supported. A source’s type resolves to one of the tokens above before the check — a JavaScript source resolves to web, and webhook and server-side SDK sources resolve to cloud. An unsupported type reports:

destination 'bigquery-prod' (type 'bq') does not support source 'my-source':
source type 'amp' is not among supported source types: android, android_kotlin, ...

The destination config must carry a connection_mode entry for that source type. This lives on the destination spec, not on the connection spec. Without it:

destination 'bigquery-prod' config has no 'connection_mode' entry for source type 'web'

Google BigQuery needs no additional config keys to connect a source of any type.

Secrets

credentials is the only secret key. Write it as a {{ .VAR }} reference and supply the value at apply time:

yaml 
config:
  credentials: "{{ .BQ_CREDENTIALS }}"
bash 
export RUDDER_BQ_CREDENTIALS="$(cat service-account.json)"
rudder-cli apply

# or
rudder-cli apply --var-file secrets.vars.yaml

Note that rudder-cli import writes credentials back as a {{ .VAR }} placeholder rather than its value, since the API doesn’t return secrets. Fill the placeholder in before the first apply.

See How to Use Variable Substitution in Rudder CLI.

See more