RudderStack Docs – profiles.yaml

Archive: ID Stitcher

Mon, 01 Jan 0001 00:00:00 +0000

The ID stitcher model maps and unifies all the specified identifiers across inputs. It tracks the user journey uniquely across all the data sources and stitches them together to a rudder_id.

When you define an ID stitcher model in profiles.yaml, you must also reference it from the corresponding entity in pb_project.yaml.

Set id_stitcher: models/<model_name> on the entity (for example, id_stitcher: models/user_id_stitcher for the user entity). If this is missing, the intended ID stitcher is not used and you may get unintended results.

Video walkthrough

Overview

You can define an ID stitcher in the pb_project.yaml file:

models:
 - name: model_name
 model_type: `id_stitcher`
 model_spec:
 entity_key: entity_name
 materialization:
 run_type: discrete/incremental
 incremental_timedelta: 96h
 edge_sources:
 - from: path/to_inputs_table
 - from: path/to_inputs_table

Field	Description
`name`	Model name.
`model_type`	Model type - set this parameter to `id_stitcher` to implement an ID stitcher.
`model_spec`	Defines the model configuration.
`entity_key`	Specifies the entity to stitch.
`run_type`	Run type - set this parameter to `discrete` or `incremental`. (Default is incremental).
`incremental_timedelta`	In incremental ID stitching, only events from the last run are typically processed to update the ID graph. However, since events can arrive in the warehouse with delays, this parameter adds a buffer period to ensure completeness. The ID stitcher will process events starting from `incremental_timedelta` before the last run timestamp, creating an overlap window to capture any delayed events. Default value: 4 days (96 hours).
`edge_sources`	List of inputs from `inputs.yaml` and `sql-models.yaml` used for identity stitching.

Example

models:
 - name: user_id_stitcher
 model_type: id_stitcher
 model_spec:
 entity_key: user
 materialization:
 run_type: incremental
 incremental_timedelta: 96h
 edge_sources:
 - from: inputs/rsIdentifies
 - from: inputs/rsTracks

Metadata logging

Two optional model spec flags control metadata logging for nodes and direct edges in the resulting Identity Graph. Both flags are supported for Snowflake and BigQuery.

Flag	Default value	Description
`log_direct_edge_info`	`false`	Populates `direct_edge_info.edges_by_type` on the per-row `METADATA` column with connected node IDs, their `first_seen_at` timestamps, and a per-source provenance array.
`log_node_metadata`	`false`	Populates `node_info` on the per-row `METADATA` column with one entry per input model that contributed the node.

models:
 - name: user_id_stitcher
 model_type: id_stitcher
 model_spec:
 entity_key: user
 log_direct_edge_info: true
 log_node_metadata: true
 edge_sources:
 - from: inputs/rsIdentifies
 - from: inputs/rsTracks

For the metadata column structure, per-source contract, and incremental-run behavior, see Metadata and observability.

When cardinality rules are enabled on the entity, the metadata column also carries cardinality_info on capped edges, and the <material>_<entity>_CARDINALITY_AUDIT table records every dropped edge along with the input model that contributed it.

Archive: Var Groups (Features)

Mon, 01 Jan 0001 00:00:00 +0000

var_groups define and organize features for entities and cohorts, and they consist of input_vars and entity_vars.

You can var_groups in the profiles.yaml file:

var_groups:
 name: var_groups_name
 entity_key: entity_name
 time_grain: day
 vars:
 - input_var:
 <input-var-config>
 - entity_var:
 <entity-var-config>

Key	Description
`name`	Name of the `var_groups`.
`entity_key`	`entity` for the defined vars.
`time_grain`	Controls how often features in this group are computed. This parameter accepts the following values: `tick`, `10minutes`, `hour`, `day`, `week`, `month`, `year`. See Timegrains for details.
`vars`	Definition of entity and input vars.
`entity_var`	Entity variable configuration.
`input_var`	Input variable configuration. See warning below.

Input vars scan all input data on every run, which can be expensive for large datasets.

Consider using entity_var with Incremental Features or Incremental SQL Models instead for better performance.

Use input_var only when you need to add a calculated column to the input table itself (for example, window functions that must be computed before entity-level aggregation).

Example

var_groups:
 name: user_var_group
 entity_key: user
 time_grain: day  # Features in this group are computed daily
 vars:
 - input_var:
 name: page_count
 select: count(distinct url)
 from: inputs/rsPages
 window:
 partition_by:
 - profile_id
 - date
 description: Add column to rsPages SQL model to get a page count per date
 - entity_var:
 name: web_dates_visited_365_days
 select: array_agg(distinct object_construct('date', date, 'page_count',{{pages_orderby_table.page_count}}))
 from: inputs/rsPages
 description: rolling 365 days array of json objects with the UTC timestamps the user visited the website along with the number of pages visited per date stamp
 - entity_var:
 name: first_order_date
 select: min(order_date)
 from: inputs/orders
 description: First order date
 - entity_var:
 name: last_order_date
 select: max(order_date)
 from: inputs/orders
 is_feature: false
 - entity_var:
 name: days_since_la`st_order
 select: "date_diff(CURRENT_DATE(), date({{user.last_order_date}}), day)"
 description: Days since user last completed an order

Archive: Define Cohorts in Profiles

Mon, 01 Jan 0001 00:00:00 +0000

You can define cohorts along with any associated features and feature views in the profiles.yaml file.

models:
 - name: cohort_name
 model_type: entity_cohort
 model_spec:
 extends: cohort/path  # Optional
 materialization: # Optional
 output_type: table\view 
 filter_expression: # Optional
 AND/OR:
 - filter_condition
 feature_views: # Optional
 - id: column
 name: feature_view_name

Key	Description
`name` Required	Model name.
`model_type` Required	Model type - set this parameter to `entity_cohort`.
`model_spec` Required	Model configuration.
`extends`	Path to the parent cohort. `users/all` is used for all members of an entity.
`output_type`	Warehouse output, either `table` or `view`.
`filter_expression`	Defines the filters.
`type`	Filter type - accepted values are `include` or `exclude`.
`value`	SQL statement of rows to filter (similar to a `where` clause).
`feature_view`	Defines the `id` and `name` of any views.

Example

# Basic cohort

models:
 - name: knownUsers
 model_type: entity_cohort
 model_spec:
 extends: user/all
 materialization:
 output_type: table
 filter_expression: "{{ user.id_type_email_count }} > 0"

# A cohort of a cohort

models:
 - name: knownUsersOrderWithCC
 model_type: entity_cohort
 model_spec:
 extends: models/knownUsers
 materialization:
 output_type: view
 filter_expression:
 AND:
 - "{{ knownUsers.has_credit_card }} = 1"
 - "{{ knownUsers.order_total }} > 0"

# Adding a feature view to a cohort

models:
 - name: knownUsers
 model_type: entity_cohort
 model_spec:
 extends: user/all
 materialization:
 output_type: table
 filter_expression: "{{ user.id_type_email_count }} > 1"
 feature_views:
 name: known_users_feature_view
 - id: email
 name: known_users_by_email

# Combining two cohorts into a new cohort

models:
 - name: americanUsers
 model_type: entity_cohort
 model_spec:
 extends: user/all
 filter_expression:
 OR:
 - "{{ warehouse.EntityCohortCondition('models/north_american_users', 'in') }}"
 - "{{ warehouse.EntityCohortCondition('models/south_american_users', 'in') }}"

# Create var_group for a cohort

var_groups:
 - name: known_users_vars
 entity_cohort: models/knownUsers
 vars:
 - entity_var:
 name: last_transaction
 select: max(order_date)
 from: inputs/orders

View cohorts in RudderStack dashboard

You can view the output for cohorts in the Entities tab of the UI once you import a Profiles project from Git and run it.

Contact the Profiles support team in RudderStack’s Community Slack if you are unable to see the Entities tab.