# Amazon S3 Destination Config Reference


> Complete Rudder CLI reference for the Amazon S3 destination config keys, source types, and secrets.



[Amazon S3](https://www.rudderstack.com/docs/destinations/streaming-destinations/amazon-s3/) is an object storage destination. RudderStack batches events and writes them as files into an S3 bucket you own.

In a S3 destination spec:

- `type: s3`
- `definition_version: 1`

## Sample configuration

```yaml
version: rudder/v1
kind: destination
metadata:
  name: amazon-s3-prod
spec:
  id: amazon-s3-prod
  display_name: Amazon S3 Production
  type: s3
  definition_version: 1
  enabled: true
  config:
    bucket_name: rudder-events-prod
    prefix: rudder/events
    role_based_auth: true
    iam_role_arn: "arn:aws:iam::123456789012:role/RudderStackS3Access"
    enable_sse: false

    connection_mode:
      web: cloud
      android_kotlin: cloud
    consent_management:
      web:
        - provider: oneTrust
          consents:
            - analytics
            - marketing
```

The above example uses role-based authentication, so it carries no access keys. Which of `iam_role_arn`, `access_key_id`, and `access_key` you set depends on `role_based_auth` — see [Authentication](#authentication).

## Config keys

`config` accepts only the keys listed below. The [shared config key rules](https://www.rudderstack.com/docs/rudder-cli/destination-types/#config-key-rules) cover unknown keys, defaults, and immutability.

### Bucket

<div class="rs-ckeys">
<div class="rs-ckey" id="bucket_name">
  <div class="rs-ckey__title">
    <p class="rs-ckey__name">bucket_name</p>Required</div>
  <p class="rs-ckey__field">
    Type:
    <code>string</code>
  </p>
<p class="rs-ckey__field">
  Description:
  Name of the S3 bucket RudderStack writes event files to. The bucket must already exist.
</p>

<div class="rs-ckey__notes">
  <p class="rs-ckey__field">Notes:</p>
  <ul>
<li>At most 100 characters, and must not contain line breaks.</li>
<li>A <code>{{ path || fallback }}</code> template is accepted in place of a literal, and isn&rsquo;t measured against the length limit.</li>
</ul>

</div>

</div>

<div class="rs-ckey" id="prefix">
  <div class="rs-ckey__title">
    <p class="rs-ckey__name">prefix</p></div>
  <p class="rs-ckey__field">
    Type:
    <code>string</code>
  </p>
<p class="rs-ckey__field">
  Description:
  Folder prefix inside the bucket. RudderStack creates a folder with this name and writes all files beneath it, at <code>s3://&lt;bucket_name&gt;/&lt;prefix&gt;/</code>.
</p>

<div class="rs-ckey__notes">
  <p class="rs-ckey__field">Notes:</p>
  <ul>
<li>At most 100 characters, and must not contain line breaks.</li>
<li>Templates are accepted on the same terms as <code>bucket_name</code>.</li>
</ul>

</div>

</div>

<div class="rs-ckey" id="enable_sse">
  <div class="rs-ckey__title">
    <p class="rs-ckey__name">enable_sse</p></div>
  <p class="rs-ckey__field">
    Type:
    <code>boolean</code>
  </p><p class="rs-ckey__field">
    Default value:
    <code>false</code>
  </p>
<p class="rs-ckey__field">
  Description:
  Enable server-side encryption. When <code>true</code>, RudderStack adds the header <code>x-amz-server-side-encryption: AES256</code> to each <code>PutObject</code> request.
</p>

<div class="rs-ckey__extra">
  See <a href="https://www.rudderstack.com/docs/destinations/streaming-destinations/amazon-s3/#s3-managed-keys" >Encryption with S3 managed keys</a>.
</div>

</div>

</div>

### Authentication

`role_based_auth` selects the authentication method, and decides which of the remaining three keys are required.

**Warning**

Rudder CLI checks only that the keys the selected method needs are present. It doesn&#39;t reject the keys belonging to the other method, so a spec carrying both an `iam_role_arn` and an access key pair passes `validate` and applies. 

Leave the unused method&#39;s keys out — otherwise you store credentials the destination never reads.

<div class="rs-ckeys">
<div class="rs-ckey" id="role_based_auth">
  <div class="rs-ckey__title">
    <p class="rs-ckey__name">role_based_auth</p>Required</div>
  <p class="rs-ckey__field">
    Type:
    <code>boolean</code>
  </p>
<p class="rs-ckey__field">
  Description:
  Whether to authenticate with an IAM role. Set it to <code>true</code> to use <code>iam_role_arn</code>, or <code>false</code> to use the access key pair.
</p>

<div class="rs-ckey__notes">
  <p class="rs-ckey__field">Notes:</p>
  <ul>
<li>The dashboard defaults this field to <code>true</code>. Rudder CLI requires it explicitly.</li>
<li>A spec that omits this key fails validation with <code>'role_based_auth' is required</code>.</li>
</ul>

</div>

</div>

<div class="rs-ckey" id="iam_role_arn">
  <div class="rs-ckey__title">
    <p class="rs-ckey__name">iam_role_arn</p>Required</div>
  <p class="rs-ckey__field">
    Type:
    <code>string</code>
  </p>
<p class="rs-ckey__field">
  Description:
  ARN of the IAM role RudderStack assumes to write to the bucket.
</p>

<div class="rs-ckey__notes">
  <p class="rs-ckey__field">Notes:</p>
  <ul>
<li>Required when <code>role_based_auth</code> is <code>true</code>. Leave it unset otherwise.</li>
<li>At most 100 characters, and must not contain line breaks.</li>
<li>Templates are accepted.</li>
</ul>

</div>

<div class="rs-ckey__extra">
  See <a href="https://www.rudderstack.com/docs/destinations/aws-iam-role-for-rudderstack/" >AWS IAM Role for RudderStack</a> for how to create the role and scope its policy.
</div>

</div>

<div class="rs-ckey" id="access_key_id">
  <div class="rs-ckey__title">
    <p class="rs-ckey__name">access_key_id</p>RequiredSecret</div>
  <p class="rs-ckey__field">
    Type:
    <code>string</code>
  </p>
<p class="rs-ckey__field">
  Description:
  AWS access key ID authorizing RudderStack to write to the bucket.
</p>

<div class="rs-ckey__notes">
  <p class="rs-ckey__field">Notes:</p>
  <ul>
<li>Required when <code>role_based_auth</code> is <code>false</code>. Leave it unset otherwise.</li>
</ul>

</div>

<div class="rs-ckey__extra">
  Supply it as a <code>{{ .VAR }}</code> reference rather than a literal — see <a href="#secrets" >Secrets</a>.
</div>

</div>

<div class="rs-ckey" id="access_key">
  <div class="rs-ckey__title">
    <p class="rs-ckey__name">access_key</p>RequiredSecret</div>
  <p class="rs-ckey__field">
    Type:
    <code>string</code>
  </p>
<p class="rs-ckey__field">
  Description:
  AWS secret access key matching <code>access_key_id</code>.
</p>

<div class="rs-ckey__notes">
  <p class="rs-ckey__field">Notes:</p>
  <ul>
<li>Required when <code>role_based_auth</code> is <code>false</code>.</li>
</ul>

</div>

<div class="rs-ckey__extra">
  <strong>Warning:</strong> RudderStack recommends role-based authentication. The access key method is deprecated and will be discontinued. See <a href="https://www.rudderstack.com/docs/destinations/streaming-destinations/amazon-s3/#permissions" >Permissions</a>.
</div>

</div>

</div>

**Info:** Either method needs a bucket policy granting RudderStack write access. Role-based authentication on its own doesn&#39;t grant it.

### Per-source keys

Both keys are objects keyed by the local source type — the tokens listed under [Source types](#source-types). A key naming a source type this destination doesn't support fails validation.

<div class="rs-ckeys">
<div class="rs-ckey" id="connection_mode">
  <div class="rs-ckey__title">
    <p class="rs-ckey__name">connection_mode</p></div>
  <p class="rs-ckey__field">
    Type:
    <code>object</code>
  </p>
<p class="rs-ckey__field">
  Description:
  Maps each source type you connect to the mode its events reach S3 in, using the modes in <a href="#source-types" >Source types</a>.
</p>

<div class="rs-ckey__notes">
  <p class="rs-ckey__field">Notes:</p>
  <ul>
<li>An entry is required for each source type you connect — see <a href="#connect-a-source" >Connect a source</a>.</li>
</ul>

</div>

<div class="rs-ckey__extra">
  <div class="rs-code">
  <div class="rs-code__head">yaml<button class="rs-code__copy" type="button">
      
      Copy
    </button>
  </div>
  <div class="highlight"><pre class="chroma"><code class="language-yaml" data-lang="yaml">connection_mode:
  web: cloud
  android_kotlin: cloud</code></pre></div>
</div>
</div>

</div>

<div class="rs-ckey" id="consent_management">
  <div class="rs-ckey__title">
    <p class="rs-ckey__name">consent_management</p></div>
  <p class="rs-ckey__field">
    Type:
    <code>object</code>
  </p>
<p class="rs-ckey__field">
  Description:
  Consent provider configuration per source type. The entry shape, accepted providers, and the rules on <code>resolution_strategy</code> and <code>consents</code> are shared across all destination types — see <a href="https://www.rudderstack.com/docs/rudder-cli/destination-types/#consent-management" >Consent management</a>.
</p>

</div>

</div>

## Source types

Amazon S3 accepts events from these source types in the mentioned connection modes:

| Source type | Connection mode |
| :-----| :-----|
| `android` | `cloud` |
| `android_kotlin` | `cloud` |
| `ios` | `cloud` |
| `ios_swift` | `cloud` |
| `web` | `cloud` |
| `unity` | `cloud` |
| `react_native` | `cloud` |
| `flutter` | `cloud` |
| `cordova` | `cloud` |
| `cloud` | `cloud` |
| `warehouse` | `cloud` |

Every source type is `cloud` only — events reach the bucket from RudderStack's servers, never in device mode.

`warehouse` is the token a [Reverse ETL source](https://www.rudderstack.com/docs/sources/reverse-etl/) resolves to. Rudder CLI manages the destination’s `warehouse` entry — if the destination is connected to a Reverse ETL source, set `connection_mode.warehouse: cloud` in its spec, or `apply` removes it.

Rudder CLI can also declare the Reverse ETL connection itself, for the types listed under [Supported destinations](https://www.rudderstack.com/docs/rudder-cli/yaml-retl-connections/#supported-destinations) — see [Manage Reverse ETL Connections using Rudder CLI](https://www.rudderstack.com/docs/rudder-cli/retl-connections/).

**Info:** The dashboard additionally offers S3 to AMP and Shopify sources. Rudder CLI doesn&#39;t manage those connections, so `amp` and `shopify` are invalid here.

## Connect a source

An event stream connection to this destination is checked against two rules at `validate` time.

**The source's type must be supported.** A source's type resolves to one of the tokens above before the check — a JavaScript source resolves to `web`, and webhook and server-side SDK sources resolve to `cloud`. Every type an event stream source can resolve to is supported here, so this check always passes.

**The destination config must carry a `connection_mode` entry for that source type.** This lives on the destination spec, not on the [connection spec](https://www.rudderstack.com/docs/rudder-cli/yaml-connections/). Without it:

```text
destination 'amazon-s3-prod' config has no 'connection_mode' entry for source type 'web'
```

Amazon S3 needs no additional config keys to connect a source of any type.

## Secrets

`access_key_id` and `access_key` are the secret keys, and apply only when `role_based_auth` is `false`. Write each as a `{{ .VAR }}` reference and supply the value at apply time:

```yaml
config:
  bucket_name: rudder-events-prod
  role_based_auth: false
  access_key_id: "{{ .AWS_ACCESS_KEY_ID }}"
  access_key: "{{ .AWS_SECRET_ACCESS_KEY }}"
```

```bash
export RUDDER_AWS_ACCESS_KEY_ID="AKIAIOSFODNN7EXAMPLE"
export RUDDER_AWS_SECRET_ACCESS_KEY="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
rudder-cli apply

# or
rudder-cli apply --var-file secrets.vars.yaml
```

Note that:

- `rudder-cli import` writes only the secrets the API returns, as `{{ .VAR }}` references, and leaves the others out. Add any secret key it leaves out, then supply the variables before you apply — see [Secrets](https://www.rudderstack.com/docs/rudder-cli/destination-types/#secrets).
- `iam_role_arn` isn't a secret — an ARN identifies a role but grants nothing on its own.

See [How to Use Variable Substitution in Rudder CLI](https://www.rudderstack.com/docs/rudder-cli/variable-substitution/).

## Changes by release

<table>
  <thead>
    <tr>
      <th>Version</th>
      <th>Change</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>v0.29.0</td>
      <td><strong>Breaking:</strong> A <code>null</code> config value, written as <code>key: null</code>, <code>key: ~</code>, or <code>key:</code> with no value, fails <code>validate</code> with <code>'&lt;key&gt;' must not be null; set a value or remove the key</code>. Earlier releases accepted it: a top-level null planned an update on every <code>apply</code>, and a null inside a list item failed at the API.<br><strong>What to do:</strong> Remove the key to leave it unset, or give it a value.</td>
    </tr>
    <tr>
      <td>v0.29.0</td>
      <td><strong>Changed:</strong> <code>validate</code> warns when a secret key holds anything but one <code>{{ .VAR }}</code> reference with no default, and when one variable fills two secret keys. It also masks secret values in the lines it prints.</td>
    </tr>
    <tr>
      <td>v0.29.0</td>
      <td><strong>Changed:</strong> <code>rudder-cli import</code> writes each secret reference single-quoted, as <code>'{{ .VAR }}'</code>, so a value containing double quotes, like a JSON key file, substitutes as a string. A single quote in a value that fills such a reference is written as <code>''</code>.<br><strong>What to do:</strong> In a spec imported by an earlier release, quote each reference to a JSON value by hand: <code>'{{ .VAR }}'</code>.</td>
    </tr>
    <tr>
      <td>v0.29.0</td>
      <td><strong>Changed:</strong> <code>validate</code> reports a value of the wrong type once per key, such as <code>'email' must be a string</code>, instead of one raw decoding error.</td>
    </tr>
    <tr>
      <td>v0.28.0</td>
      <td><strong>Added:</strong> <code>warehouse</code> source type, for destinations connected to a Reverse ETL source. Set <code>connection_mode.warehouse: cloud</code> for such a destination, or <code>apply</code> removes the entry.</td>
    </tr>
  </tbody>
</table>

For changes across every destination type, see [Destination Type Changes by Release](https://www.rudderstack.com/docs/rudder-cli/destination-types/changes/).

## See more

- [Amazon S3 Destination](https://www.rudderstack.com/docs/destinations/streaming-destinations/amazon-s3/) for bucket setup, permissions, and the file format RudderStack writes
- [Destination Type Reference for Rudder CLI](https://www.rudderstack.com/docs/rudder-cli/destination-types/) for the rules shared across destination types
- [Destination YAML Reference](https://www.rudderstack.com/docs/rudder-cli/yaml-destinations/) for the spec envelope