# Identity Graph

This guide introduces you to the concept of identity graphs in Profiles and shows how to use them in your Profiles project.

## Overview

Multiple identities is the biggest obstacle to getting a full picture of your customers, user, or any other [`entity`]({{< ref "profiles/concepts/entities.md" >}}).

Companies gather user data across digital touchpoints like websites, mobile apps, enterprise systems like CRMs, marketing platforms, etc. During this process, a single user is identified with multiple identifiers across your systems, like email address, phone number, device ID, anonymous ID, user name, etc. To create a unified user profile, it is vital to collect and stitch all of those different identifiers into one canonical identifier. 

You can leverage Profiles' `id_stitcher` model to take your inputs and stitch all the identifiers together into an identity graph.

An identity graph consists of two main components:

- **Nodes**: Represent individual identifiers that belong to an entity.
- **Edges**: Represent the relationships observed between these identifiers in your input data.

## Requirements

- Defined [`id_types`]({{< ref "profiles/dev-docs/pb-project-yaml/id-types.md" >}}) for your entity and mapped to your input sources.
- Identifiers from your inputs must be of the string data type. If the columns in your input sources are not of string types already, you must cast them explicitly.

## Usage

The identity graph is located in the [`profiles.yaml`]({{< ref "profiles/dev-docs/profiles-yaml/" >}}) file as an `id_stitcher` model.

```yaml
models:
  - name: user_id_stitcher
    model_type: id_stitcher
    model_spec:
      entity_key: user
      edge_sources:
        - from: inputs/rsIdentifies
        - from: inputs/rsPages
```

### Link the identity graph to your entity

After you define an `id_stitcher` model in [`profiles.yaml`]({{< ref "profiles/dev-docs/profiles-yaml/" >}}), you must reference it from the corresponding entity in your [`pb_project.yaml`]({{< ref "profiles/dev-docs/pb-project-yaml/" >}}) file. Set the `id_stitcher` key on the entity to the model path (for example, `models/user_id_stitcher` for a model named `user_id_stitcher`).

{{< warning >}}
**Always set `id_stitcher` on each entity that has a custom ID stitcher model defined in `profiles.yaml`.**

If the entity does not specify `id_stitcher`, the intended id stitcher is not used and you may get unintended results.
{{< /warning >}}

Example entity configuration in `pb_project.yaml`:

```yaml
entities:
  - name: user
    id_stitcher: models/user_id_stitcher
    id_types:
      - user_id
      - anonymous_id

... # Rest of the configuration
```

See [Entity]({{< ref "profiles/dev-docs/pb-project-yaml/entities.md" >}}) for full entity configuration options.

## Metadata and observability

To provide transparency into the stitching process, you can enable metadata logging using the following flags in your `id_stitcher` model specification. These flags control what metadata is logged for the identity graph components:

| Flag | Description |
| :-----| :-----|
| `log_direct_edge_info` | Logs metadata about the direct relationships (edges) between identifiers. |
| `log_node_metadata`| Logs metadata for each individual identifier (node). |

Both flags are supported on **Snowflake** and **BigQuery**. Other supported warehouses do not currently emit the `METADATA` column.

An example of how to enable these flags is shown below:

```yaml
models:
  - name: user_id_stitcher
    model_type: id_stitcher
    model_spec:
      entity_key: user
      log_direct_edge_info: true
      log_node_metadata: true
      edge_sources:
        - from: inputs/rsIdentifies
        - from: inputs/rsPages
```

This metadata helps you understand how identifiers are connected and trace the lineage of your identity graph.

### Metadata structure

The generated metadata contains two main objects:

| Object | Details |
| :----| :----|
| `node_info`| Array of objects containing aggregated metadata for the node (identifier) itself. It includes the `source_model` where the identifier was first seen and the `first_seen_at` timestamp. |
| `direct_edge_info` | Object describing the direct relationships (edges) of a node. The `edges_by_type` field is an **array** of buckets — one bucket per identifier type. Each bucket contains a `count` of edges and an `edges` array of the connected nodes, with the per-source provenance of each edge. |

The following example shows one node with a single email edge from one input model:

```json
{
  "direct_edge_info": {
    "edges_by_type": [
      {
        "count": 1,
        "edges": [
          {
            "first_seen_at": "2022-01-01 10:00:00.000",
            "id": "s1_email@test.com",
            "sources": [
              {
                "first_seen_at": "2022-01-01 10:00:00.000",
                "source_model": "events_app"
              }
            ]
          }
        ],
        "type": "email"
      }
    ]
  },
  "node_info": [
    {
      "first_seen_at": "2022-01-01 10:00:00.000",
      "source_model": [
        "events_app"
      ]
    }
  ]
}
```

When the same edge is observed in more than one input model, each contributing model appears as a separate entry in the edge's `sources` array. The edge-level `first_seen_at` is the earliest timestamp across all sources:

```json
{
  "direct_edge_info": {
    "edges_by_type": [
      {
        "count": 1,
        "edges": [
          {
            "first_seen_at": "2022-02-10 08:00:00.000",
            "id": "ms_email@test.com",
            "sources": [
              {
                "first_seen_at": "2022-02-15 10:00:00.000",
                "source_model": "events_app"
              },
              {
                "first_seen_at": "2022-02-10 08:00:00.000",
                "source_model": "events_web"
              }
            ]
          }
        ],
        "type": "email"
      }
    ]
  },
  "node_info": [
    {
      "first_seen_at": "2022-02-15 10:00:00.000",
      "source_model": ["events_app"]
    },
    {
      "first_seen_at": "2022-02-10 08:00:00.000",
      "source_model": ["events_web"]
    }
  ]
}
```

{{< info >}}
**Incremental runs do not merge metadata across runs**

On each incremental run, the per-node `METADATA` column reflects only the source attribution observed during the current run's data window. If a node was contributed by `events_app` in a previous run and only by `events_mobile` in the current run, the metadata column shows `events_mobile` alone — the previous run's source list is not carried forward.

This applies to both `node_info[].source_model` and `direct_edge_info.edges_by_type[].edges[].sources[]`. The stitched ID graph itself (the `id` and `other_id` columns) does carry forward across incremental runs as expected; only the metadata column resets per run.
{{< /info >}}

### Reading the metadata column

The `METADATA` column is a per-row JSON document attached to every row of the ID stitcher output. Each row describes one identifier (`other_id`) and how it was observed during the current run.

The top-level structure is fixed:

- `node_info` is a deduplicated array, one entry per input model that contributed the node. Each entry carries the `first_seen_at` timestamp for that model and a one-element `source_model` array.
- `direct_edge_info.edges_by_type` is an array of buckets. Each bucket has a `type` (the ID type of the connected nodes), a `count`, and an `edges` array.
- Each edge in `edges` has the connected node's `id`, the edge-level `first_seen_at`, and a `sources` array. The `sources` array carries one entry per contributing input model with that model's own `first_seen_at`.

#### Multi-source edges

The same edge can appear in multiple inputs. The number of entries in an edge's `sources` array shows how many inputs reported the edge in this run — the `source_model` and `first_seen_at` per entry trace the edge back to a specific table and time. 

A single entry in `sources` means only one input model contributed the edge in this run — not that the edge is weaker.

#### Incremental run behavior

Two incremental runs of the same project can produce different `sources` arrays for the same edge, depending on which inputs landed new rows in each run. 

When you compare metadata across runs, treat the `sources` array as a per-run snapshot, not as lifetime history. To preserve cross-run source history, snapshot the column into a downstream table on each run.

For more information on how violator edges interact with the metadata column under cardinality rules, see [ID Graph Cardinality Rules]({{< ref "profiles/dev-docs/advanced-id-stitcher-features/id-graph-cardinality-rules.md" >}}).

### Metadata size and limits

Each row's `METADATA` JSON is bounded by structure, not by data scale. Two internal caps keep each row's JSON small even for highly-connected nodes:

| Field | Limit | What it means |
| :--- | :--- | :--- |
| `direct_edge_info.edges_by_type` length | **10 buckets** | At most 10 distinct identifier types per node appear in the metadata. |
| `edges_by_type[].edges` length | **10 edges per bucket** | At most 10 direct neighbors of a given type are listed per node. |
| `edges[].sources` length | Less than or equal to the number of `edge_sources:` in your project YAML | One entry per contributing input model. |
| `node_info` length | Less than or equal to the number of `edge_sources:` in your project YAML | One entry per contributing input model. |

The two hardcoded limits are not currently configurable from YAML. In practice, the defaults are large enough for almost all production identity graphs. If you lose meaningful detail, consider whether [ID Graph Cardinality Rules]({{< ref "profiles/dev-docs/advanced-id-stitcher-features/id-graph-cardinality-rules.md" >}}) on the entity can tighten the underlying graph rather than expanding the metadata caps.

{{< info >}}
**Detecting edges hidden by the cap**

Each `edges_by_type[]` bucket reports the **uncapped** total in its `count` field, while the `edges` array is capped at 10. A row where `count` exceeds the length of `edges` means the metadata column shows a truncated view of that node's neighbors. 

You can run this check directly against the `METADATA` column without consulting the cardinality audit table — it complements the `cardinality_info` field used for rule-capped edges. See [ID Graph Cardinality Rules]({{< ref "profiles/dev-docs/advanced-id-stitcher-features/id-graph-cardinality-rules.md" >}}) for the rule-capped case.
{{< /info >}}

## Best practices

- For the `identity_stitcher` model, use **only** the `id_types` and `inputs` necessary to stitch the identifiers relevant to an `entity`. For example, if you don’t care about anonymous traffic, do not add `anonymous_id` as an `id_type` or any `inputs` related to anonymous traffic.
- Conceptually, you can think of your `identity_graph` table as a kind of fact table in star or Snowflake schema. All of your core identifiers are in the center table and are used to connect to your `inputs`.
- Only include `inputs` as edge sources that include `id_types` directly related to the entity (the first level out from the fact table in a Snowflake schema). For example, you may need to include an `id_type` for product names because they are in a separate table but you do not want to include them in the `identity_graph`.

<br />