RudderStack Docs – Profiles Overview

Archive: How Profiles Works

Mon, 01 Jan 0001 00:00:00 +0000

Profiles has two primary functions:

Generating an identity graph.
Computing features on top of that identity graph.

Generating an identity graph

Known and unknown users

At a high level, customer data can be categorized in to data from both known and unknown users.

You can track unknown users at an individual level using the RudderStack SDKs and capture their anonymousId. However, you don’t have any additional information about these users. Known users are identified by some additional information, often traits like email, phone, etc.

These IDs live in many places across your data warehouse or data lake, most often as a column (or columns) in individual tables that are loaded from various data sources.

You can use Profiles to stitch together both known and unknown IDs into a single table. The IDs are linked using an autogenerated ID, called the rudderId, which is a canonical identifier for each user.

Use case

Tying together known and unknown IDs for a user is foundational for ensuring accurate customer data. Consider the following customer journey:

A user browsed your website anonymously six months ago.
The user returns three months later and signs up for your newsletter.
One month later, the user clicks an ad on their phone and visits your website anonymously through their mobile browser, and makes a purchase.

In many cases, the data generated by this journey would be counted as two or three separate users, depending on the tools used to track the behavior. Profiles ID stitching ties the data together, revealing that all of those activities were performed by the same user. Profiles represents these data points as different nodes and edges of the identity graph.

Computing features to build a customer 360 table

The identity graph serves as a map of where each user’s data lives across tables in your data warehouse. Leveraging this map, Profiles makes it easy for you to pull and compute features for each user across all of your data.

For example, a user might have purchase events spread across multiple tables: one for mobile, one from a previous platform, and one from your new eCommerce site. With Profiles, you can easily sum those values across tables for a single user to understand lifetime value.

Features are output to a table, which you can use as your single source of truth for customer 360.

Archive: Profiles Quickstart

Mon, 01 Jan 0001 00:00:00 +0000

This guide walks you through creating a Profiles project using the Profile Builder (PB) tool.

Prerequisites

Python 3 installed on your computer.
Admin privileges on your computer.

Required permissions

Admins have full access to manage Profiles projects in RudderStack.
Members can have the following permissions depending on their workspace policy:

Permission	Description
Edit	Make changes to the configuration of Profiles projects
Create & Delete	Create or delete Profiles projects in the workspace
Connect	Connect a Profiles project to a warehouse destination The Connect permission also applies on the warehouse source used to create the Profiles project.

Click here to see how these permissions appear in the workspace policy.

To make a connection, that is, connect a Profiles project to a warehouse destination, the member must have both Edit and Connect permissions on both the resources.

Permissions for legacy RBAC system

In the legacy Permissions Management (RBAC) system:

Org Admins and members with the Connections Admin role in their workspace policy can create and delete Profiles projects, manage their configuration, and connect them to a warehouse destination
Members with the Connections Editor role in their workspace policy can only edit the Profiles project configuration and connect Profiles projects to a warehouse destination

1. Install Profile Builder

Create and activate the virtual environment:

python3 -m venv .venv
source .venv/bin/activate

Install the Profiles Builder (PB) package:

pip3 install profiles-rudderstack

Verify the installation was successful:

pb version

2. Create warehouse connection

Profiles supports Snowflake, Redshift, BigQuery and Databricks. The user in your connection configuration must have permission to read from schemas containing source data and write to the schema created for Profiles.

Create a warehouse connection:

pb init connection

Follow the prompts to enter details about your warehouse connection:

Select a warehouse: s
Enter Connection Name: my_warehouse
Enter target: (default:dev): # Press enter, leaving it to default
Enter account: company.us-east-1
Enter warehouse: RUDDER_WAREHOUSE
Enter dbname: RUDDER_EVENTS
Enter schema: RUDDER_PROFILES
Enter user: <YOUR_USERNAME> // Optional for Databricks
Enter password: <YOUR_PASSWORD>
Enter role: <YOUR_ROLE>
Append to /Users/<user_name>/.pb/siteconfig.yaml? [y/N] y

This creates a local site configuration file inside your home directory: ~/.pb/siteconfig.yaml. Your Profiles project uses this file to access the warehouse, git credentials, and other details. If you don’t see the file, enable the view hidden files option.

3. Create Profiles project

Initialize a new Profiles project:

pb init pb-project -o my-project

This creates a new project in the my-project folder with the initial project structure.

If you would like to create a project in the current folder instead, run:

pb init pb-project -o .

Open the pb_project.yaml file at the root of your project and set the value of connection: to the connection name as defined in the previous step.

4. Add inputs

Open the inputs.yaml file in the models folder. This file is where you will define your inputs.

Each input has a name, source table (in the format database.schema.table), occurred_at_col (timestamp column if available), and list of entity ids. Each ID inclues a select (column name), type (ID type as defined in pb_project.yaml) and entity name (defined in pb_project.yaml).

Edit the two sample inputs to use your own data. This may only require changing the table fields to point to your identifies and tracks tables. If your identifies table does not include email addresses, remove that ID. If email address is included under a different column name, change the select field accordingly.

5. Validate warehouse access

Validate your warehouse connection has the necessary access:

pb validate access

If there are no errors, proceed to the next step. In case of errors, make sure to provide the necessary warehouse permissions.

6. Run project

Run the project on your data warehouse:

pb run

This compiles your project into SQL files stored in the output folder, then runs that SQL on your warehouse.

7. Add feature

Open the profiles.yaml file in the models folder. There should be a var group with four entity vars, two of which are features (included in feature views).

Add a new entity var / feature that gets the event count for each user:

- entity_var:
 name: event_count
 select: COUNT(*)
 from: inputs/rsTracks
 description: No. of events tracked for customer

Profiles automatically groups data by customer, so you only need to specify the aggregate function.

If you run the project again, you will see this event_count feature in your feature views.

8. Add cohort

The feature views in your warehouse contain features for all customers. A cohort can be used to create feature views for a subset of customers.

In the profiles.yaml file, add a cohort for power users:

models:
 - name: power_users
 model_type: entity_cohort
 model_spec:
 extends: user/all
 materialization:
 output_type: view
 filter_pipeline:
 - type: include
 value: "{{ user.event_count }} > 45"

This power_users cohort uses the event_count entity var defined above, only including customers with more than 45 events tracked.

Feature views for this cohort will be automatically generated during the next project run.

Archive: Create Profiles Project in the RudderStack dashboard

Mon, 01 Jan 0001 00:00:00 +0000

This guide lists the steps to create a Profiles project in the RudderStack dashboard.

Create Profiles project

Log in to the RudderStack dashboard and go to Unify > Profiles option in the left sidebar.
Click Create project.

Select Web-based editor (IDE) on the next screen.

You can also import an existing project from your Git repository.

Enter a unique name and description for your Profiles project.
Select a data warehouse from the dropdown.

RudderStack currently supports the following warehouses for creating a Profiles project:

Snowflake

Redshift

Databricks

BigQuery

Enter the schema name for the corresponding warehouse in the Schema field where you want your Profiles project to store the outputs.
Click Create Profiles Project and select Open in editor mode.
Enter the schema name you want to use for development purposes for your editing session and click Continue.

RudderStack recommends using a development schema while editing your project so that you can modify your YAML configuration files, run your changes within the editor, and validate the outputs in a separate development schema.

This makes it convenient to test your changes in the development schema without affecting the outputs in your production schema.

You will be able to see the following screen displaying a sample Profiles project with different YAML files:

Edit the YAML files and save and commit the changes to your project.
Click the Run button to run the project. Upon successful completion, you can view the outputs in your data warehouse.

Convert to a Git project

Once you set up a Profiles project in the dashboard, you can convert it to a configuration-based Git repository by uploading it either to GitHub, GitLab, or Bitbucket:

Navigate to the Unify > Profiles option in the left sidebar to view all your Profiles projects.
Click Convert to Git button next to the project you want to convert.
Create a new Git repository.
Enter the SSH URL of your Git repository to be used for your Profiles project and click Continue.
Copy the SSH public key and click Add deploy key which will take you to your git repository’s Deploy keys section. See deploy keys section in the GitHub, GitLab, or Bitbucket documentation for more information.

Make sure your Git repository has at least one commit for successful validation.

Click Add deploy key, add a Title, paste your SSH public key, and click Add key.
Click Continue to let RudderStack verify read access.
Click Download project to download the Profiles project you set up in the dashboard.
Commit the downloaded project to your Git repository.
Once the project is committed successfully, return to the dashboard and click Convert Profiles project.

Start/Stop project run

Once created, you can:

Run your Profiles project using either of the following ways:
- Clicking Run in the History tab of the project.
- Programmatically using the Profiles API.
Stop a running project by clicking the red button in the Status column:

It might take up to a few seconds to stop a running project.

Download project

To download the Profiles project, click the arrow icon corresponding to your Profiles project and click Download this project:

Once downloaded, you can view the project folder structure, modify the files, or run various commands to execute the desired use-cases.

Project details

To view the Profiles project details, click the arrow icon corresponding to your Profiles project:

Option	Description
Entities	Lists the entities, cohorts, features, activations, etc. for your Profiles project
History	Displays the history of Profile runs
Settings	Displays your profile settings and lets you delete your Profiles project. You can edit the project by clicking the edit icon next to each section
Alerts	Set up Alert thresholds for the Profiles project

Profile details

You can also view the details of a specific profile in your Profiles project:

In your Profiles project’s Entities tab, click View button across the entity for which you want to see the profile:
Click Profile Lookup tab to search a profile record.
Type an available unique identifier like email, phone number, user id, anonymous id, or the entity’s main_id (for example, user_main_id) and click Search user profile.

Note that:

It can take a few minutes for the data preview to show up in your profile’s History tab.

If you keep getting a blank screen, it may be because you do not have sufficient access. Make sure you have the relevant permissions with the ability to access PII. In case the problem persists, contact RudderStack support.