Amazon S3 Destination Config Reference Beta

Amazon S3 is an object storage destination. RudderStack batches events and writes them as files into an S3 bucket you own.

In a S3 destination spec:

• type: s3
• definition_version: 1

Sample configuration

version: rudder/v1
kind: destination
metadata:
  name: amazon-s3-prod
spec:
  id: amazon-s3-prod
  display_name: Amazon S3 Production
  type: s3
  definition_version: 1
  enabled: true
  config:
    bucket_name: rudder-events-prod
    prefix: rudder/events
    role_based_auth: true
    iam_role_arn: "arn:aws:iam::123456789012:role/RudderStackS3Access"
    enable_sse: false

    connection_mode:
      web: cloud
      android_kotlin: cloud
    consent_management:
      web:
        - provider: oneTrust
          consents:
            - analytics
            - marketing

The above example uses role-based authentication, so it carries no access keys. Which of iam_role_arn, access_key_id, and access_key you set depends on role_based_auth — see Authentication.

Config keys

config accepts only the keys listed below. The shared config key rules cover unknown keys, defaults, and immutability.

Bucket

Authentication

role_based_auth selects the authentication method, and decides which of the remaining three keys are required.

Rudder CLI checks only that the keys the selected method needs are present. It doesn’t reject the keys belonging to the other method, so a spec carrying both an iam_role_arn and an access key pair passes validate and applies.

Leave the unused method’s keys out — otherwise you store credentials the destination never reads.

Either method needs a bucket policy granting RudderStack write access. Role-based authentication on its own doesn’t grant it.

Per-source keys

Both keys are objects keyed by the local source type — the tokens listed under Source types. A key naming a source type this destination doesn’t support fails validation.

connection_mode:
  web: cloud
  android_kotlin: cloud

Source types

Amazon S3 accepts events from these source types in the mentioned connection modes:

Every source type is cloud only — events reach the bucket from RudderStack’s servers, never in device mode.

The dashboard additionally offers S3 to AMP, Shopify, and Reverse ETL sources. Rudder CLI doesn’t manage those connections, so amp, shopify, and warehouse are invalid here.

Connect a source

An event stream connection to this destination is checked against two rules at validate time.

The source’s type must be supported. A source’s type resolves to one of the tokens above before the check — a JavaScript source resolves to web, and webhook and server-side SDK sources resolve to cloud. An unsupported type reports:

destination 'amazon-s3-prod' (type 's3') does not support source 'my-source':
source type 'amp' is not among supported source types: android, android_kotlin, ...

The destination config must carry a connection_mode entry for that source type. This lives on the destination spec, not on the connection spec. Without it:

destination 'amazon-s3-prod' config has no 'connection_mode' entry for source type 'web'

Amazon S3 needs no additional config keys to connect a source of any type.

Secrets

access_key_id and access_key are the secret keys, and apply only when role_based_auth is false. Write each as a {{ .VAR }} reference and supply the value at apply time:

config:
  bucket_name: rudder-events-prod
  role_based_auth: false
  access_key_id: "{{ .AWS_ACCESS_KEY_ID }}"
  access_key: "{{ .AWS_SECRET_ACCESS_KEY }}"

export RUDDER_AWS_ACCESS_KEY_ID="AKIAIOSFODNN7EXAMPLE"
export RUDDER_AWS_SECRET_ACCESS_KEY="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
rudder-cli apply

# or
rudder-cli apply --var-file secrets.vars.yaml

Note that:

• A {{ .VAR }} reference satisfies the requirement check, so a spec using role-free authentication validates before the values are supplied.
• rudder-cli import writes both keys back as {{ .VAR }} placeholders rather than their values, since the API doesn’t return secrets. Fill the placeholders in before the first apply.
• iam_role_arn isn’t a secret — an ARN identifies a role but grants nothing on its own.

See How to Use Variable Substitution in Rudder CLI.

See more

• Amazon S3 Destination for bucket setup, permissions, and the file format RudderStack writes
• Destination Type Reference for Rudder CLI for the rules shared across destination types
• Destination YAML Reference for the spec envelope