Confluent Cloud Destination Config Reference

Confluent Cloud is a streaming destination — a fully managed Apache Kafka service. RudderStack publishes each event as a JSON message to a topic in your Confluent Cloud cluster, keyed by the event’s userId, or anonymousId when userId is absent, so one user’s events stay in the same partition.

In a Confluent Cloud destination spec:

• type: confluent_cloud
• definition_version: 1

Sample configuration

version: rudder/v1
kind: destination
metadata:
  name: confluent-cloud-prod
spec:
  id: confluent-cloud-prod
  display_name: Confluent Cloud Production
  type: confluent_cloud
  definition_version: 1
  enabled: true
  config:
    bootstrap_server: pkc-a1b2c.us-east-1.aws.confluent.cloud:9092
    topic: rudder-events
    api_key: "{{ .CONFLUENT_API_KEY }}"
    api_secret: "{{ .CONFLUENT_API_SECRET }}"

    connection_mode:
      web: cloud
      cloud: cloud
    consent_management:
      web:
        - provider: oneTrust
          consents:
            - analytics

The above example keeps the API key pair out of the YAML with {{ .VAR }} references — see Secrets. Every key in it except the per-source keys is required.

Config keys

config accepts only the keys listed below. The shared config key rules cover unknown keys, defaults, and immutability.

Confluent Cloud has no topic routing or Avro settings. For those, or for a self-managed cluster, see Apache Kafka.

Connection

Authentication

RudderStack connects over TLS and authenticates with SASL/PLAIN, using api_key as the username and api_secret as the password.

Per-source keys

Both keys are objects keyed by the local source type — the tokens listed under Source types. A key naming a source type this destination doesn’t support fails validation.

connection_mode:
  web: cloud
  cloud: cloud

Source types

Confluent Cloud accepts events from these source types in the mentioned connection modes:

Every source type is cloud only — events reach Confluent Cloud from RudderStack’s servers, never in device mode.

warehouse is the token a Reverse ETL source resolves to. Rudder CLI manages the destination’s warehouse entry — if the destination is connected to a Reverse ETL source, set connection_mode.warehouse: cloud in its spec, or apply removes it.

Rudder CLI can also declare the Reverse ETL connection itself, for the types listed under Supported destinations — see Manage Reverse ETL Connections using Rudder CLI.

The dashboard additionally offers Confluent Cloud to AMP and Shopify sources. Rudder CLI doesn’t manage those connections, so amp and shopify are invalid here.

Connect a source

An event stream connection to this destination is checked against two rules at validate time.

The source’s type must be supported. A source’s type resolves to one of the tokens above before the check — a JavaScript source resolves to web, and webhook and server-side SDK sources resolve to cloud. Every type an event stream source can resolve to is supported here, so this check always passes.

The destination config must carry a connection_mode entry for that source type. This lives on the destination spec, not on the connection spec. Without it:

destination 'confluent-cloud-prod' config has no 'connection_mode' entry for source type 'web'

Confluent Cloud needs no additional config keys to connect a source of any type.

Secrets

api_key and api_secret are the secret keys. Write each as a {{ .VAR }} reference and supply the value at apply time:

config:
  api_key: "{{ .CONFLUENT_API_KEY }}"
  api_secret: "{{ .CONFLUENT_API_SECRET }}"

export RUDDER_CONFLUENT_API_KEY="..."
export RUDDER_CONFLUENT_API_SECRET="..."
rudder-cli apply

# or
rudder-cli apply --var-file secrets.vars.yaml

Note that:

• rudder-cli import writes only the secrets the API returns, as {{ .VAR }} references, and leaves the others out. Add any secret key it leaves out, then supply the variables before you apply — see Secrets.

See How to Use Variable Substitution in Rudder CLI.

Changes by release

No changes since this type became available.

For changes across every destination type, see Destination Type Changes by Release.

See more

• Confluent Cloud Destination for the connection settings and the partition key
• Apache Kafka Destination Config Reference for a self-managed Kafka cluster
• Destination Type Reference for Rudder CLI for the rules shared across destination types
• Destination YAML Reference for the spec envelope