MaxCompute Destination

The MaxCompute warehouse destination is in Private Beta. Contact Customer Success to enable it for your workspace.

Use MaxCompute as a warehouse destination to load events through an Alibaba Cloud Object Storage Service (OSS) bucket. RudderStack stages files in OSS before loading them into your MaxCompute project. For common event-to-table mappings, see Warehouse Schema; MaxCompute-specific differences are noted below.

Prepare your project and OSS bucket

1. Enable Enable Schema for the MaxCompute project so it supports the three-tier project, schema, and table namespace. This upgrade is one-way and has a settling period; wait for it to finish before creating the destination. Plan the change with your MaxCompute administrator.
2. Check that the project uses the V2.0 data type edition. MaxCompute V1.0 can’t represent TIMESTAMP; a session hint doesn’t upgrade the project. Arrange a project edition upgrade before loading data if needed.
3. Create or select an OSS bucket in the same region as the MaxCompute project. Record the project region and endpoint. The configured endpoint must be exactly https://service.<region>.maxcompute.aliyun.com/api, with <region> matching the project’s region. Use the same region for OSS.
4. Choose a new schema name for Namespace, but don’t pre-create the schema. RudderStack creates it on the first sync. This version can’t load into a schema owned by another principal.
5. Choose an OSS directory path containing the bucket name and, optionally, a directory prefix, for example, rudder-staging or rudder-staging/rudder/events. Use that same prefix when scoping OSS access. There is no separate prefix setting.

Grant minimum-privilege access

Use role authentication unless you need AccessKey authentication. Both modes need MaxCompute project membership and a separate read role that MaxCompute assumes to load objects from OSS. Replace angle-bracketed placeholders below with the indicated RudderStack or Alibaba Cloud values.

Grant project permissions for role authentication

1. Create an access role for RudderStack in RAM. In its trust policy, allow the RudderStack RAM principal acs:ram::5332290435279528:user/rudderstack-maxcompute-caller-prod and require sts:ExternalId to equal your RudderStack workspace ID. Replace <RUDDERSTACK_WORKSPACE_ID> in this example with your workspace ID. Record the access role ARN for the MaxCompute account setup.

   json Copy

   {
     "Version": "1",
     "Statement": [
       {
         "Effect": "Allow",
         "Principal": {
           "RAM": ["acs:ram::5332290435279528:user/rudderstack-maxcompute-caller-prod"]
         },
         "Action": "sts:AssumeRole",
         "Condition": {
           "StringEquals": {
             "sts:ExternalId": "<RUDDERSTACK_WORKSPACE_ID>"
           }
         }
       }
     ]
   }

2. Grant the access role OSS access to the configured bucket and directory prefix.

3. As a MaxCompute project administrator, add the access role as a project user and grant it the following four project permissions. Replace <account-uid> with your Alibaba Cloud account UID (not the workspace ID or the RAM user’s UID). Replace <role-name> with the name of the access role, not the read role:

ADD USER `RAM$<account-uid>:role/<role-name>`;
GRANT CreateInstance ON PROJECT <project-name> TO USER `RAM$<account-uid>:role/<role-name>`;
GRANT CreateSchema ON PROJECT <project-name> TO USER `RAM$<account-uid>:role/<role-name>`;
GRANT CreateTable ON PROJECT <project-name> TO USER `RAM$<account-uid>:role/<role-name>`;
GRANT List ON PROJECT <project-name> TO USER `RAM$<account-uid>:role/<role-name>`;

Grant project permissions for AccessKey authentication

1. Create an API-only RAM user and generate an AccessKey for it. Record the AccessKey ID and secret securely. Find the user’s numeric RAM user ID, labeled UID in the RAM console; the user name isn’t a substitute.
2. Give the RAM user the same prefix-scoped OSS access as the access role.
3. As a MaxCompute project administrator, add the RAM user by numeric RAM user ID and grant it the same four project permissions. Use your Alibaba Cloud account UID for <account-uid> here as well:

ADD USER `RAM$<account-uid>:<ram-user-id>`;
GRANT CreateInstance ON PROJECT <project-name> TO USER `RAM$<account-uid>:<ram-user-id>`;
GRANT CreateSchema ON PROJECT <project-name> TO USER `RAM$<account-uid>:<ram-user-id>`;
GRANT CreateTable ON PROJECT <project-name> TO USER `RAM$<account-uid>:<ram-user-id>`;
GRANT List ON PROJECT <project-name> TO USER `RAM$<account-uid>:<ram-user-id>`;

Adding a RAM user name instead of its numeric UID fails with ODPS-0030001 … is not a valid account. A new AccessKey can also take about 15 minutes to activate; during that time, validation fails just as it does for an incorrect key. Wait for activation before troubleshooting a newly created key.

A RAM policy alone doesn’t add the role or user to the MaxCompute project. Even AliyunMaxComputeFullAccess without ADD USER fails with ODPS-0420095.

Scope OSS permissions

For the access role (or the RAM user in AccessKey mode), allow staging file writes and reads (oss:PutObject, oss:GetObject, oss:AbortMultipartUpload, and oss:ListParts) only on the configured bucket’s objects under the chosen directory prefix. For rudder-staging/rudder/events, scope the object resource to acs:oss:*:*:rudder-staging/rudder/events/*; for a bucket-only path, use acs:oss:*:*:rudder-staging/*. Allow oss:ListObjects on the bucket resource (acs:oss:*:*:rudder-staging) only with an oss:Prefix condition restricting listings to the chosen directory prefix (rudder/events/* in this example). Include the trailing slash so the policy doesn’t match sibling directories such as rudder/events-private/. For a bucket-only path, allow listing across that bucket, not other buckets. If Clean up staged files after successful sync is on, also allow oss:DeleteObject on the same object resource. Turning cleanup off removes the delete requirement. Don’t grant access to unrelated buckets or prefixes.

Create a separate read role for MaxCompute LOAD. Its RAM trust policy must trust the service principal odps.aliyuncs.com. Allow oss:GetObject on the same scoped object resource, and oss:ListObjects (restricted by the same oss:Prefix condition when a prefix is set) and oss:GetBucketInfo on that bucket resource. Enter its ARN in OSS RAM role ARN in the MaxCompute account for either account type; the access role or AccessKey user doesn’t replace it.

Configure MaxCompute in RudderStack

In the RudderStack dashboard, add MaxCompute as a warehouse destination, name it, and connect a source. During Initial setup, select or create a MaxCompute account and enter the following settings:

Namespace, skip views creation, partition column, and partition type are fixed at creation. Choose them before saving the destination; you can’t change them later.

Skip Users table and Skip Tracks table are under Configuration settings → Warehouse settings and can be changed later. Skip Users table is on by default; identify events still load into identifies. Turn it off to maintain a users table with carry-forward traits. Skip Tracks table is off by default.

Verify the connection and first sync

Run the connection test when creating the destination. It validates OSS access first, then MaxCompute access and a test load. With cleanup enabled, it checks OSS delete access. Confirm that the first sync creates the schema and tables and that rows appear in identifies after sending an identify event. Check Live Events or the destination’s Syncs tab for upload status. The test can leave one small OSS object even when the connection succeeds; see the test-file FAQ.

How RudderStack creates the schema

RudderStack creates a schema in your MaxCompute project with the name set in the Namespace field. The namespace is required for MaxCompute and can’t be changed after you create the destination.

See the Warehouse Schema guide for details on the tables and columns RudderStack creates.

RudderStack creates every MaxCompute table as an auto-partitioned table. Unless Skip views creation is on, it also creates a deduplication view for each table.

Partitioned tables

RudderStack creates auto-partitioned tables with a single partition key, pt. MaxCompute fills pt for every row by truncating the Partition column to the Partition type, so you can filter on pt to query a subset of data:

Note that:

• pt values are in UTC.
• Partition column and partition type are fixed when you create the destination.
• RudderStack reserves pt, so an event property named pt is stored in a column named _pt.
• If a later batch omits the chosen partition column, its rows still load into the __NULL__ partition. This is especially relevant for Sent at and Original timestamp, which may be absent from SDK events. Query those rows directly in the base table: the deduplication view and the users carry-forward read exclude them.
• When querying a base table, filter on pt (for example, WHERE pt >= '2026-11-01'). Many MaxCompute projects reject queries that read every partition; see How do I avoid a full-scan error?.

RudderStack does not discard duplicate data while loading it into MaxCompute.

Views

In addition to tables, RudderStack creates a view (<table_name>_view) for every table for deduplication purposes, unless Skip views creation is on. The view returns one row per event (id), keeping the most recently loaded one. This applies even when the duplicates are in different partitions.

Note that:

• RudderStack recommends using the corresponding view to avoid duplicate events in your query results.
• The view covers partitions from 60 days back to 2 days ahead, evaluated at query time in the session time zone. The lower edge can shift with the session offset. You don’t need an additional pt filter when querying a view. For older data, query the base table with a pt filter.
• RudderStack recreates the view when it loads the table, so new columns appear automatically. Don’t edit the view query: your changes are overwritten on the next sync. Create your own view instead if you need a different time window.
• Rows in the __NULL__ partition aren’t included in the view.
• For rudder_discards, the view returns one row per row_id, column_name, and table_name.

Skip views creation

Turn on Skip views creation to disable creating views in MaxCompute. Like the partition settings, it’s fixed when you create the destination.

RudderStack recommends skipping views creation only if:

• You do not need views and want to take care of deduplication yourself.
• You have security reasons to limit access to the tables.

Users table

When Skip Users table is off (it’s on by default), RudderStack keeps each user’s latest traits in the users table. On every sync, for each user identified in that sync, RudderStack merges the user’s rows from the last 60 days with the new identify calls. It takes the latest non-null value of each trait and adds one new row for the user:

Note that:

• The users table has one row per user per sync. When views are enabled, use users_view to get one row per user, with their current traits.
• Traits are carried forward only from the last 60 days. A user not identified in more than 60 days starts again from the new identify call.
• Omitting a trait or sending null does not clear it; the previous value is kept.

FAQ

How do I avoid a full-scan error?

MaxCompute can return ODPS-0130071 when a base-table query doesn’t filter on partition key pt. Filter table queries on pt, or set odps.sql.allow.fullscan=true in your session if you need a full scan. Each <table>_view already filters on pt from 60 days back to two days ahead, so querying a view without an additional pt filter doesn’t trigger this guard. RudderStack’s own queries always filter on pt and never use the hint.

Why do deleted OSS files still incur storage charges?

When OSS bucket versioning is enabled, RudderStack deletes staged files without specifying a version ID. OSS creates a delete marker, but the noncurrent version remains billable. Add an OSS lifecycle rule with NoncurrentVersionExpiration and expired-delete-marker removal to clear retained versions and markers.

Which time zone applies to timestamps and partitions?

MaxCompute projects default to UTC+8 in every region. Stored timestamp values and pt partitions are correct in UTC, but displayed query results and functions such as TO_CHAR and DATETRUNC use the session time zone. For events between 16:00 and 24:00 UTC, session-local midnight doesn’t match UTC partition midnight. Run SET odps.sql.timezone=UTC; in each query session or configure your BI tool’s session time zone. Alibaba cautions against changing the project time zone on an active project.

What are the hourly partition limits?

Two different limits apply:

• 60,000 partitions per table (about 6.8 years of hourly partitions) is table capacity. Once reached, a LOAD requiring a new partition fails. Plan retention or partition cleanup before reaching it.
• 10,000 partitions per query (about 14 months of hourly partitions for a full-history scan) limits your queries, not RudderStack loads. Filter or split queries that span more partitions.

Daily partitions have no practical ceiling for this use case. Partition type is fixed at creation, so choose hourly only if you can manage these limits.

Why does credential or OSS validation fail before the project is checked?

The first validation step checks object storage. A wrong AccessKey, or a newly created key that hasn’t activated yet, fails there before any MaxCompute project check. If your OSS directory path falls outside the policy’s object resource or listing prefix, validation returns oss_access_denied: check the OSS directory path and the policy’s resource scope. With cleanup on, also check oss:DeleteObject access.

Why does validation fail with “MaxCompute schema is not owned by the current principal”?

RudderStack only loads into a schema owned by the identity it connects as. If you change the destination’s MaxCompute account to a different identity (for example, from Access key to RAM role, or to a different role or RAM user), the existing schema still belongs to the previous identity, and validation fails. Rotating the AccessKey of the same RAM user isn’t affected. To switch identity, transfer the schema’s ownership to the new identity in MaxCompute, or create a new destination with a new namespace.

Why is there a rudder-test-payload/ object in OSS?

Each connection test leaves one small rudder-test-payload/… object under your configured OSS directory path from the load-table check. This also happens with other RudderStack warehouse destinations. It’s safe to delete.

Can I query nested objects as JSON documents?

No. Nested objects flatten into table columns, rather than a native JSON document. Property-heavy events can exhaust RudderStack’s 1,200-column limit even though a single event may carry up to 1,600 columns. Plan event properties to keep the table’s column count within the limit.

For general warehouse questions, see Warehouse Destinations FAQ.