# BigQuery Stream Destination Config Reference


> Complete Rudder CLI reference for the BigQuery Stream destination config keys, deduplication, source types, and secrets.



[BigQuery Stream](https://www.rudderstack.com/docs/destinations/streaming-destinations/bigquery-stream/) streams events into an existing BigQuery table through Google's streaming insert API, so rows are queryable within seconds rather than after a scheduled load.

In a BigQuery Stream destination spec:

- `type: bqstream`
- `definition_version: 1`

## Sample configuration

```yaml
version: rudder/v1
kind: destination
metadata:
  name: bigquery-stream-prod
spec:
  id: bigquery-stream-prod
  display_name: BigQuery Stream Production
  type: bqstream
  definition_version: 1
  enabled: true
  config:
    project_id: acme-analytics
    dataset_id: product_events
    table_id: product_inserts
    insert_id: productId
    credentials: '{{ .BQSTREAM_CREDENTIALS }}'

    connection_mode:
      web: cloud
      cloud: cloud
    consent_management:
      web:
        - provider: oneTrust
          consents:
            - analytics
```

The above example deduplicates on the `productId` event property. Leave `insert_id` out to stream without deduplication — see [Deduplication](#deduplication).

## Config keys

`config` accepts only the keys listed below. The [shared config key rules](https://www.rudderstack.com/docs/rudder-cli/destination-types/#config-key-rules) cover unknown keys, defaults, and immutability.

**Info:** BigQuery Stream accepts only `track` events, and RudderStack forwards each event's properties without fetching or checking the table schema. Make sure the `track` payload matches the columns of the table named in `table_id` — see [Send events to BigQuery Stream](https://www.rudderstack.com/docs/destinations/streaming-destinations/bigquery-stream/#send-events-to-bigquery-stream).

### Table

<div class="rs-ckeys">
<div class="rs-ckey" id="project_id">
  <div class="rs-ckey__title">
    <p class="rs-ckey__name">project_id</p>Required</div>
  <p class="rs-ckey__field">
    Type:
    <code>string</code>
  </p>
<p class="rs-ckey__field">
  Description:
  GCP project ID that holds the dataset.
</p>

</div>

<div class="rs-ckey" id="dataset_id">
  <div class="rs-ckey__title">
    <p class="rs-ckey__name">dataset_id</p>Required</div>
  <p class="rs-ckey__field">
    Type:
    <code>string</code>
  </p>
<p class="rs-ckey__field">
  Description:
  ID of the dataset, within <code>project_id</code>, that holds the table.
</p>

</div>

<div class="rs-ckey" id="table_id">
  <div class="rs-ckey__title">
    <p class="rs-ckey__name">table_id</p>Required</div>
  <p class="rs-ckey__field">
    Type:
    <code>string</code>
  </p>
<p class="rs-ckey__field">
  Description:
  ID of the table RudderStack streams events into. The table must already exist.
</p>

</div>

</div>

### Deduplication

<div class="rs-ckeys">
<div class="rs-ckey" id="insert_id">
  <div class="rs-ckey__title">
    <p class="rs-ckey__name">insert_id</p></div>
  <p class="rs-ckey__field">
    Type:
    <code>string</code>
  </p>
<p class="rs-ckey__field">
  Description:
  Name of the event property whose value BigQuery uses as the <code>insertId</code> to deduplicate rows — for example <code>productId</code>. It names the property, not the value.
</p>

<div class="rs-ckey__notes">
  <p class="rs-ckey__field">Notes:</p>
  <ul>
<li>The property&rsquo;s value must be a number or a string.</li>
<li>Deduplication applies only when the value is present in the event&rsquo;s <code>properties</code>.</li>
</ul>

</div>

<div class="rs-ckey__extra">
  <p>To pick the property per event, use a template with a fallback:</p>
<div class="rs-code">
  <div class="rs-code__head">yaml<button class="rs-code__copy" type="button">
      
      Copy
    </button>
  </div>
  <div class="highlight"><pre class="chroma"><code class="language-yaml" data-lang="yaml">insert_id: &#39;{{ message.uniqueId || &#34;productId&#34; }}&#39;</code></pre></div>
</div><p>See <a href="https://www.rudderstack.com/docs/destinations/streaming-destinations/bigquery-stream/#configure-insertid-dynamically" >Configure insertId dynamically</a> for how each case resolves.</p>

</div>

</div>

</div>

### Authentication

<div class="rs-ckeys">
<div class="rs-ckey" id="credentials">
  <div class="rs-ckey__title">
    <p class="rs-ckey__name">credentials</p>RequiredSecret</div>
  <p class="rs-ckey__field">
    Type:
    <code>string</code>
  </p>
<p class="rs-ckey__field">
  Description:
  Contents of the JSON key file for a GCP service account that can insert rows into the table.
</p>

<div class="rs-ckey__extra">
  <p>Supply it as a <code>{{ .VAR }}</code> reference rather than a literal — see <a href="#secrets" >Secrets</a>.</p>
<p>See <a href="https://www.rudderstack.com/docs/destinations/streaming-destinations/bigquery-stream/#create-a-service-account" >Create a service account</a> for the steps.</p>

</div>

</div>

</div>

### Per-source keys

Both keys are objects keyed by the local source type — the tokens listed under [Source types](#source-types). A key naming a source type this destination doesn't support fails validation.

<div class="rs-ckeys">
<div class="rs-ckey" id="connection_mode">
  <div class="rs-ckey__title">
    <p class="rs-ckey__name">connection_mode</p></div>
  <p class="rs-ckey__field">
    Type:
    <code>object</code>
  </p>
<p class="rs-ckey__field">
  Description:
  Maps each source type you connect to the mode its events reach BigQuery in, using the modes in <a href="#source-types" >Source types</a>.
</p>

<div class="rs-ckey__notes">
  <p class="rs-ckey__field">Notes:</p>
  <ul>
<li>An entry is required for each source type you connect — see <a href="#connect-a-source" >Connect a source</a>.</li>
</ul>

</div>

<div class="rs-ckey__extra">
  <div class="rs-code">
  <div class="rs-code__head">yaml<button class="rs-code__copy" type="button">
      
      Copy
    </button>
  </div>
  <div class="highlight"><pre class="chroma"><code class="language-yaml" data-lang="yaml">connection_mode:
  web: cloud
  cloud: cloud</code></pre></div>
</div>
</div>

</div>

<div class="rs-ckey" id="consent_management">
  <div class="rs-ckey__title">
    <p class="rs-ckey__name">consent_management</p></div>
  <p class="rs-ckey__field">
    Type:
    <code>object</code>
  </p>
<p class="rs-ckey__field">
  Description:
  Consent provider configuration per source type. The entry shape, accepted providers, and the rules on <code>resolution_strategy</code> and <code>consents</code> are shared across all destination types — see <a href="https://www.rudderstack.com/docs/rudder-cli/destination-types/#consent-management" >Consent management</a>.
</p>

</div>

</div>

## Source types

BigQuery Stream accepts events from these source types in the mentioned connection modes:

| Source type | Connection mode |
| :-----| :-----|
| `android` | `cloud` |
| `android_kotlin` | `cloud` |
| `ios` | `cloud` |
| `ios_swift` | `cloud` |
| `web` | `cloud` |
| `unity` | `cloud` |
| `react_native` | `cloud` |
| `flutter` | `cloud` |
| `cordova` | `cloud` |
| `cloud` | `cloud` |
| `warehouse` | `cloud` |

Every source type is `cloud` only — events reach the table from RudderStack's servers, never in device mode.

`warehouse` is the token a [Reverse ETL source](https://www.rudderstack.com/docs/sources/reverse-etl/) resolves to. Rudder CLI manages the destination’s `warehouse` entry — if the destination is connected to a Reverse ETL source, set `connection_mode.warehouse: cloud` in its spec, or `apply` removes it.

Rudder CLI can also declare the Reverse ETL connection itself, for the types listed under [Supported destinations](https://www.rudderstack.com/docs/rudder-cli/yaml-retl-connections/#supported-destinations) — see [Manage Reverse ETL Connections using Rudder CLI](https://www.rudderstack.com/docs/rudder-cli/retl-connections/).

**Info:** The dashboard additionally offers BigQuery Stream to AMP and Shopify sources. Rudder CLI doesn&#39;t manage those connections, so `amp` and `shopify` are invalid here.

## Connect a source

An event stream connection to this destination is checked against two rules at `validate` time.

**The source's type must be supported.** A source's type resolves to one of the tokens above before the check — a JavaScript source resolves to `web`, and webhook and server-side SDK sources resolve to `cloud`. Every type an event stream source can resolve to is supported here, so this check always passes.

**The destination config must carry a `connection_mode` entry for that source type.** This lives on the destination spec, not on the [connection spec](https://www.rudderstack.com/docs/rudder-cli/yaml-connections/). Without it:

```text
destination 'bigquery-stream-prod' config has no 'connection_mode' entry for source type 'web'
```

BigQuery Stream needs no additional config keys to connect a source of any type.

## Secrets

`credentials` is the only secret key. Write it as a `{{ .VAR }}` reference and supply the value at apply time:

```yaml
config:
  credentials: '{{ .BQSTREAM_CREDENTIALS }}'
```

```bash
export RUDDER_BQSTREAM_CREDENTIALS="$(cat service-account.json)"
rudder-cli apply

# or
rudder-cli apply --var-file secrets.vars.yaml
```

Note that:

- `credentials` is JSON, so write its reference in single quotes, as above. Rudder CLI pastes the value into the YAML before parsing it, and the JSON's double quotes would end a double-quoted string.
- `rudder-cli import` writes only the secrets the API returns, as `{{ .VAR }}` references, and leaves the others out. Add any secret key it leaves out, then supply the variables before you apply — see [Secrets](https://www.rudderstack.com/docs/rudder-cli/destination-types/#secrets).

See [How to Use Variable Substitution in Rudder CLI](https://www.rudderstack.com/docs/rudder-cli/variable-substitution/).

## Changes by release

<table>
  <thead>
    <tr>
      <th>Version</th>
      <th>Change</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>v0.29.0</td>
      <td><strong>Breaking:</strong> A <code>null</code> config value, written as <code>key: null</code>, <code>key: ~</code>, or <code>key:</code> with no value, fails <code>validate</code> with <code>'&lt;key&gt;' must not be null; set a value or remove the key</code>. Earlier releases accepted it: a top-level null planned an update on every <code>apply</code>, and a null inside a list item failed at the API.<br><strong>What to do:</strong> Remove the key to leave it unset, or give it a value.</td>
    </tr>
    <tr>
      <td>v0.29.0</td>
      <td><strong>Changed:</strong> <code>validate</code> warns when a secret key holds anything but one <code>{{ .VAR }}</code> reference with no default, and when one variable fills two secret keys. It also masks secret values in the lines it prints.</td>
    </tr>
    <tr>
      <td>v0.29.0</td>
      <td><strong>Changed:</strong> <code>rudder-cli import</code> writes each secret reference single-quoted, as <code>'{{ .VAR }}'</code>, so a value containing double quotes, like a JSON key file, substitutes as a string. A single quote in a value that fills such a reference is written as <code>''</code>.<br><strong>What to do:</strong> In a spec imported by an earlier release, quote each reference to a JSON value by hand: <code>'{{ .VAR }}'</code>.</td>
    </tr>
    <tr>
      <td>v0.29.0</td>
      <td><strong>Changed:</strong> <code>validate</code> reports a value of the wrong type once per key, such as <code>'email' must be a string</code>, instead of one raw decoding error.</td>
    </tr>
    <tr>
      <td>v0.28.0</td>
      <td><strong>Added:</strong> <code>warehouse</code> source type, for destinations connected to a Reverse ETL source. Set <code>connection_mode.warehouse: cloud</code> for such a destination, or <code>apply</code> removes the entry.</td>
    </tr>
  </tbody>
</table>

For changes across every destination type, see [Destination Type Changes by Release](https://www.rudderstack.com/docs/rudder-cli/destination-types/changes/).

## See more

- [BigQuery Stream Destination](https://www.rudderstack.com/docs/destinations/streaming-destinations/bigquery-stream/) for table schema requirements and deduplication cases
- [Destination Type Reference for Rudder CLI](https://www.rudderstack.com/docs/rudder-cli/destination-types/) for the rules shared across destination types
- [Destination YAML Reference](https://www.rudderstack.com/docs/rudder-cli/yaml-destinations/) for the spec envelope