# ID Graph Cardinality Rules


This guide explains how to use cardinality rules to limit the maximum number of connections between different ID types in your identity graph.

{{< info >}}
Cardinality rules are supported for **Snowflake** and **BigQuery**. 

Rule enforcement, the `<material>_<entity>_CARDINALITY_AUDIT` table, and the per-edge `cardinality_info` metadata are emitted on both warehouses.
{{< /info >}}

## Overview

ID Graph cardinality rules let you define maximum connection limits between different ID types in your identity graph, providing fine-grained control over how identities are stitched together.

{{< info >}}
Cardinality rules help prevent identity explosion and maintain data quality by limiting how many connections a node of one type can have to nodes of another type. 

This is particularly useful for maintaining data quality when dealing with shared identifiers like email addresses, device IDs, or anonymous IDs that might be used across multiple users.
{{< /info >}}

Cardinality rules work seamlessly with other Profiles features:

- Input [filters]({{< ref "profiles/dev-docs/pb-project-yaml/id-types.md" >}}) reduce data volume before graph processing. The cardinality enforcement step comes after this in the pipeline, so if expected nodes are missing from your graph, input filters can be the reason.
- Manual [exclusion/inclusion rules]({{< ref "profiles/dev-docs/advanced-id-stitcher-features/id-graph-rules.md" >}}) provide explicit control over specific connections and happen after the cardinality enforcement step in the pipeline. If you find identity clusters that violate cardinality rules despite having the rules configured, those extra edges might be coming from manual inclusion overrides.

### Key features

- **Data quality**: You can prevent identity graphs from becoming overly connected due to shared or recycled identifiers.
- **Performance**: Maintain optimal graph size by controlling edge proliferation.
- **Compliance**: Enforce business rules about identity relationships.
- **Transparency**: Get full audit trail of which edges were filtered and why.
- **Incremental consistency**: Historical violations persist across runs for reliable behavior.

### Performance considerations

Consider the following when implementing cardinality rules:

- **Initial runs** may experience increased runtime as the system processes and enforces the new rules across your entire dataset.
- **Incremental runs** should see minimal performance impact once the rules are established.
- **Trade-off evaluation**: The increased runtime in initial runs delivers more precise ID stitching and better data quality. Evaluate whether the improved accuracy justifies the additional processing time for your use case.

## How cardinality rules work

### Directional rules

Cardinality rules are **directional** — they apply from a source ID type to a target ID type. For example:

- `user_id → email: max_edges = 2` means each `user_id` can connect to at most 2 `email`.
- Note that this does not limit how many `user_id` an `email` can connect to (unless you add a separate rule)

### Rule enforcement

When a rule is violated:

1. **All edges** from the violating source are removed (not just the excess ones)
2. **Nodes are preserved** as isolated stubs to maintain graph integrity
3. **Violations are audited** with full details about what was removed and why
4. **Historical context** is maintained across incremental runs

### Incremental behavior

The feature works seamlessly with incremental runs:

- **Historical violations persist**: If a node violated a rule previously, new edges to that node type are automatically blocked.
- **New violations break all edges**: If a new edge causes a violation, all current edges from that source are removed.
- **Rule removal**: If you remove a rule from your configuration, the entire id graph is built from scratch, so historical violations for that rule are no longer enforced.

## Configuration

You can add cardinality rules to your `pb_project.yaml` under the `id_types` section, as shown:

```yaml
entities:
  - name: user
    id_stitcher: models/customer_id_graph
    id_types:
      - user_id
      - email
      - anonymous_id

id_types:
  - name: user_id
    maximum_edges:
      - email: 2
      - anonymous_id: 1
  - name: email
    maximum_edges:
      - user_id: 1
  - name: anonymous_id
    maximum_edges:
      - email: 1
      - user_id: 1
```

### Configuration limits

The following limits ensure optimal performance for incremental runs:

- **Maximum edges per rule**: 10 (The `maximum_edges` value for each rule cannot exceed 10)
- **Maximum target types per source**: 5 (The `maximum_edges` list for each source ID type cannot have more than 5 entries)

{{< info >}}
The above limits only apply when using cardinality rules. Without these rules, the regular ID stitcher can connect unlimited edges, as usual.
{{< /info >}}

### Examples

This section provides examples of how to configure cardinality rules for different use cases:

{{< tabs tabTotal="2" >}}
{{% tab tabName="Ecommerce platform" %}}
```yaml
id_types:
  - name: user_id
    maximum_edges:
      - email: 2          # Users can have up to 2 emails
      - device_id: 5       # Users can have up to 5 devices
  - name: email
    maximum_edges:
      - user_id: 1         # Each email belongs to only 1 user
  - name: device_id
    maximum_edges:
      - user_id: 2         # Shared devices (family tablets)
```
{{% /tab %}}
{{% tab tabName="Marketing platform" %}}
```yaml
id_types:
  - name: anonymous_id
    maximum_edges:
      - email: 1           # Anonymous sessions link to 1 email
  - name: email
    maximum_edges:
      - user_id: 1         # Emails don't get shared between users
```
{{% /tab %}}
{{< /tabs >}}

## How rule application works

This section explains how cardinality rules are applied to your identity graph.

### Fresh and incremental runs

Cardinality rules behave differently depending on whether you're running a fresh or incremental run:

- **Fresh runs**: Rules are applied to all incoming data. If a node stays within its limits, all edges are preserved.
- **Incremental runs**: Rules consider historical violations. If a node previously violated a rule or if new data causes a violation, all edges from that source are removed, including previously valid ones.

#### Fresh run

```yaml
# Input data
user_1 → email_a
user_1 → email_b  # Valid: within limit of 2

# Result: Both edges allowed
```

#### Incremental run

```yaml
# Previous run: user_1 had 2 emails (at limit)
# New data: 
user_1 → email_c  # This causes violation

# Result: ALL edges from user_1 are broken (including previous ones)
```

### Example

Given these rules:

```yaml
id_types:
  - name: user_id
    maximum_edges:
      - email: 2
  - name: email  
    maximum_edges:
      - user_id: 1
```

And the following data:

```yaml
user_1 ↔ email@example.com
user_2 ↔ email@example.com  
user_3 ↔ email@example.com
```

**What happens**

1. `email@example.com` connects to 3 user_ids, violating the `email → user_id: 1` rule
2. **All three connections** are broken, not just the excess ones.
3. All nodes are preserved as isolated stubs.
4. Three audit entries are created documenting the violation.

## Monitoring and debugging

This section explains how to monitor and debug cardinality rules.

### Cardinality Audit table

Each violation is logged in the cardinality audit table, which resides in the same schema as the ID Stitcher output tables. The table name is derived by appending the suffix `_CARDINALITY_AUDIT` to the name of the ID Stitcher model.

The audit table is created on both Snowflake and BigQuery and contains the following columns:

| Column | Description |
|--------|-------------|
| `run_id` | ID of the run when the violation occurred. |
| `model_hash` | Hash of the model configuration. |
| `id1` | Source ID value. |
| `id1_type` | Source ID type. |
| `id2` | Target ID value. |
| `id2_type` | Target ID type. |
| `reason` | Always `"CARDINALITY_VIOLATION"`. |
| `rule_details` | JSON with `max_edges` and `current_count`. |
| `source_models` | JSON array (BigQuery) or native ARRAY (Snowflake). Each element is `{source_model, first_seen_at}` and records every input model that contributed the dropped edge, with the earliest timestamp at which that model first observed it. |

#### Sample rows

The following rows are excerpted from a `qa_metadata_validation_user_CARDINALITY_AUDIT` table after a run where two users (`viol_user` and `mviol_user`) each exceeded a `user_id → email: max_edges = 2` rule. `mviol_user` saw one of the violator emails (`mviol_shared@test.com`) from two different input models, which is why its `source_models` array has two entries:

```text
id1,        id1_type, id2,                    id2_type, model_hash, reason,                rule_details,                              run_id, source_models
viol_user,  user_id,  viol_a@test.com,        email,    ffffe585,   CARDINALITY_VIOLATION, {"current_count":3,"max_edges":2},         1,      [{"source_model":"events_app","first_seen_at":"2022-03-01 10:00:00.000"}]
viol_user,  user_id,  viol_b@test.com,        email,    ffffe585,   CARDINALITY_VIOLATION, {"current_count":3,"max_edges":2},         1,      [{"source_model":"events_app","first_seen_at":"2022-03-01 10:00:01.000"}]
viol_user,  user_id,  viol_c@test.com,        email,    ffffe585,   CARDINALITY_VIOLATION, {"current_count":3,"max_edges":2},         1,      [{"source_model":"events_app","first_seen_at":"2022-03-01 10:00:02.000"}]
mviol_user, user_id,  mviol_shared@test.com,  email,    ffffe585,   CARDINALITY_VIOLATION, {"current_count":3,"max_edges":2},         1,      [{"source_model":"events_app","first_seen_at":"2022-04-01 10:00:00.000"},{"source_model":"events_web","first_seen_at":"2022-03-25 08:00:00.000"}]
```

### `cardinality_info` in the metadata column

When edge metadata logging is enabled (see [Metadata and observability]({{< ref "profiles/concepts/identity-graph.md#metadata-and-observability" >}})), the per-node `METADATA` column on the ID stitcher output reflects rule enforcement in two ways:

- **Capped edges** for non-violator nodes carry a `cardinality_info` field on the stub edge, recording the `max_edges` and `current_count` that caused the cap.
- **Violator nodes** (nodes that exceeded a rule and had all their edges removed) appear with an **empty** `direct_edge_info.edges_by_type` array. Their `node_info` is preserved so the node still exists as an isolated stub in the graph.

For example, both a single-source violator (`viol_user`) and a multi-source violator (`mviol_user`) collapse to the same shape in the metadata column — only `node_info` survives:

```json
{
  "direct_edge_info": {
    "edges_by_type": []
  },
  "node_info": [
    {
      "first_seen_at": "2022-04-01 10:00:00.000",
      "source_model": ["events_app"]
    },
    {
      "first_seen_at": "2022-03-25 08:00:00.000",
      "source_model": ["events_web"]
    }
  ]
}
```

Because dropped edges no longer appear in the metadata column, the audit table's `source_models` column is the only surviving trace of which input model contributed each violator edge. See [Interpret the audit table](#6-interpret-the-audit-table) for a debugging workflow.

## Best practices

This section provides best practices for configuring the cardinality rules correctly in your Profiles project.

#### 1. Start conservatively

Begin with lower limits and gradually increase based on your data patterns:

```yaml
# Start here
id_types:
  - name: email
    maximum_edges:
      - user_id: 1

# Adjust based on audit results
id_types:
  - name: email
    maximum_edges: 
      - user_id: 2  # If you see legitimate shared emails
```

#### 2. Monitor audit tables

Make sure to review the Cardinality Audit table regularly to understand:

- Which rules are triggering most often
- Whether legitimate connections are being blocked
- If limits need adjustment

#### 3. Test with historical data

Before deploying rules to production:

- Run rules against historical data in a test environment.
- Analyze the audit results.
- Adjust limits based on findings.

#### 4. Consider bidirectional rules

Make sure to consider both directions of a relationship:

```yaml
# Often you want both directions
id_types:
  - name: user_id
    maximum_edges:
      - email: 2        # Users can have multiple emails
  - name: email
    maximum_edges:
      - user_id: 1      # But emails belong to one user
```

#### 5. Add rules for shared identifiers

Cardinality rules help when a single identifier can legitimately appear across many users but you don't want the graph to merge those users. Common cases:

- **Shared corporate emails** such as `info@`, `support@`, or `hr@` addresses that show up in many sessions.
- **Bot or test accounts** that share a fixed `anonymous_id` or `device_id` across thousands of synthetic users.
- **Sentinel and placeholder values** such as `0`, `unknown`, `n/a`, or a default device identifier that slipped past upstream filters and would otherwise stitch unrelated users into one cluster.

If a few well-known sentinel values are responsible for most of the spread, prefer adding them to your input [filters]({{< ref "profiles/dev-docs/pb-project-yaml/id-types.md" >}}) so they never enter the graph in the first place. Cardinality rules are the right tool when the offending values are not known in advance and you need the graph itself to flag them.

#### 6. Interpret the audit table

To triage cardinality violations:

1. **Group by `reason` and `model_hash`** to confirm the violations belong to the rule you expect. `rule_details` carries the configured `max_edges` value and the `current_count` the source node reached, so a row with `current_count: 12, max_edges: 2` is qualitatively different from one with `current_count: 3, max_edges: 2`.
2. **Sort by `id1` to find the worst offenders.** A node that appears as `id1` many times is over-connected on one or more target types; the count of distinct `id2` values for that `id1` reflects how many edges triggered the violation in the current run.
3. **Use `source_models` to trace each dropped edge back to its input.** This is the only surviving source-attribution trace for violator edges, because the per-node `direct_edge_info.edges_by_type` array is emptied when a node violates a rule. If a single `id2` carries multiple `source_models` entries, multiple input models independently reported the same edge — useful for deciding whether the offending edge is a data-quality problem in one source or a real cross-source pattern.
4. **Cross-reference with the metadata column.** For non-violator nodes that share an `id2` with a violator, look at the `cardinality_info` field on the stub edge to see whether the rule capped the edge or whether it survived intact.

#### 7. Trace source attribution in the metadata column

For nodes that did not violate a rule, the metadata column carries full source attribution per edge. The `direct_edge_info.edges_by_type[].edges[].sources[]` array lists every input model that contributed the edge, along with the timestamp each one first observed it. Combined with the audit table for violators, these two sources provide complete source-attribution coverage across the graph.

See [Metadata and observability]({{< ref "profiles/concepts/identity-graph.md#metadata-and-observability" >}}) for the full per-source contract.

## Troubleshooting

| Issue | Solution |
| :----| :-----|
| Legitimate connections being blocked | Review audit table and increase limits if needed. |
| Identity graph still too connected | Add more restrictive rules or lower existing limits. |

### Debugging

- **Check rule configuration** in `pb_project.yaml`.
- **Query audit table** to see what is blocked.
- **Review stub metadata** in the final ID graph.
- Compare **fresh vs. incremental** run results.

## Migration guide

This section provides tips on managing cardinality rules in your Profiles models.

### Enable rules on existing models

1. **Add rules gradually**: Start with one rule and observe effects
2. **Test in development**: Use a copy of production data to validate
3. **Monitor after deployment**: Watch audit tables and graph metrics
4. **Adjust as needed**: Fine-tune limits based on real-world results

### Disable rules

To stop enforcing a rule, simply remove it from your configuration:

```yaml
# Before
id_types:
  - name: email
    maximum_edges:
      - user_id: 1

# After (rule removed)
id_types:
  - name: email
    # maximum_edges section removed
```
{{< info >}}
This triggers a complete fresh run of the model. Historical violations are no longer enforced, but previous audit records remain for reference.
{{< /info >}}