Profile Builder CLI

Install the Profile Builder tool by running the following command:

pip3 install profiles-rudderstack

If you have already installed PB, use the following command to update its version:

pip3 install profiles-rudderstack -U

A sample connection for a Snowflake account is as follows:

Enter Connection Name: test
Enter target:  (default:dev):  # Press enter, leaving it to default
Enter account: ina13147.us-east-1
Enter warehouse: rudder_warehouse
Enter dbname: your_rudderstack_db 
Enter schema: rs_profiles # A segregated schema for storing tables/views created by Profiles
Enter user: profiles_test_user
Do you want to use key-pair authentication? [y/N] y
--- If you select yes ----
Enter file path containing value for privateKey: /<path>/key.pem
Enter passphrase (leave blank if private key not encrypted):
-- If you select no----
Enter password: <password>
--common for both--
Enter role: profiles_role
Append to /Users/<user_name>/.pb/siteconfig.yaml? [y/N] y

• Connection Name: Name of the connection in the project file.
• Target: Environment name, such as dev, prod, test, etc. You can specify any target name and create a separate connection for the same.
• Account: Name of your Snowflake account. Based on your cloud platform and region, you might need to append .aws, .gcp, or .azure in your account name. See Snowflake documentation for more information.
• Warehouse: Name of the warehouse.
• Database name: Name of the database inside warehouse where model outputs will be written.
• Schema: Name of the schema inside database where you’ll store identity stitcher and entity features.
• User: Name of the user in data warehouse.
• Use Snowflake key-pair authentication: RudderStack supports Snowflake’s key-pair authentication mechanism to validate your Snowflake connection.

For enhanced security, RudderStack recommends using the key pair authentication over the basic authentication mechanism (username and password).

See the following sections in the Snowflake documentation to generate and use the key pair:

• Generate private key
• Generate public key
• Assign public key to Snowflake user

If you opt for Snowflake key-pair authentication, then enter the following settings:

• Private key: Enter the file path containing the private key, for example, /<path>/<private_key>.pem.
• Passphrase: Specify the password you set while encrypting the private key. Leave this field blank if your private key is not encrypted.

The user authentication will fail if your private key is encrypted and you do not specify the passphrase.

If you opt for username and password authentication, then enter the following settings:

• Password: Password for the above user.

Then, continue with the connection setup by specifying the below settings:

• Role: Name of the user role.
• Append to site configuration file: Determines whether to add the connection details to the siteconfig.yaml file.

RudderStack supports various user authentication mechanisms for Redshift:

Enter Connection Name: test
Enter target:  (default:dev):  # Press enter, leaving it to default
Enter dbname: your_rudderstack_db 
Enter schema: rs_profiles # A segregated schema for storing tables/views created by Profiles
Enter user: profiles_test_user
Enter sslmode: options - [disable require]: disable # Enter "require" in case your Redshift connection mandates sslmode. 
How would you like to authenticate with the warehouse service? Please select your method by entering the corresponding number:
y
[1] Warehouse Credentials: Log in using your username and password.
        Format: [Username, Password]
[2] AWS Programmatic Credentials: Authenticate using one of the following AWS credentials methods:
        a) Direct input of AWS Access Key ID, Secret Access Key, and an optional Session Token.
                Format: [AWS Access Key ID, Secret Access Key, Session Token (Optional)]
        b) AWS configuration profile stored on your system.
                Format: [AWS Configuration Profile Name]
        c) Use an AWS Secrets Manager ARN to securely retrieve credentials.
                Format: [Secret ARN]

• Connection Name: Name of the connection in the project file.
• Target: Environment name, such as dev, prod, test, etc. You can specify any target name and create a separate connection for the same.
• Host: Log in to AWS Console and go to Clusters to know about host.
• Port: Port number to connect to the warehouse.
• Database name: Name of the database inside warehouse where model outputs will be written.
• Schema: Name of the schema inside database where you’ll store identity stitcher and entity features.
• User: Name of the user in data warehouse.
• Password: Password for the above user.

Warehouse credentials

If you choose to use the warehouse credentials (option 1), enter the following details:

Enter host: warehouseabc.us-west-1.redshift.amazonaws.com
Enter port: 5439
Enter password: <password>
Enter tunnel_info: Do you want to use SSH Tunnel to connect with the warehouse? (y/n): y
Enter ssh user: user
Enter ssh host: 1234
Enter ssh port: 12345
Enter ssh private key file path: /Users/alex/.ssh/id_ed25519.pub
Append to /Users/alex/.pb/siteconfig.yaml? [y/N] y

AWS Programmatic Credentials

If you choose to use AWS Access Key ID, Secret Access Key, and an optional Session Token from AWS Programmatic Credentials (option 2a), enter the following details:

For Redshift Cluster

Which Redshift Service you want to connect with?
        [c]Redshift Cluster
        [s]Redshift Serverless
c
Enter Redshift Cluster Identifier: cluster-id
Enter access_key_id: aid
Enter secret_access_key: ***
Enter session_token: If you are using temporary security credentials, please specify the session token, otherwise leave it empty.
stoken
Append to /Users/alex/.pb/siteconfig.yaml? [y/N] y

For Redshift Serverless

Which Redshift Service you want to connect with?
        [c]Redshift Cluster
        [s]Redshift Serverless
s
Enter Redshift Serverless Workgroup Name: wg-name
Enter access_key_id: aid
Enter secret_access_key: ***
Enter session_token: If you are using temporary security credentials, please specify the session token, otherwise leave it empty.
Append to /Users/alex/.pb/siteconfig.yaml? [y/N] y

If you choose to use AWS configuration profile stored on your system from AWS Programmatic Credentials (option 2b), enter the following details:

For Redshift Cluster

Which Redshift Service you want to connect with?
        [c]Redshift Cluster
        [s]Redshift Serverless
c
Enter Redshift Cluster Identifier: ci   
Enter shared_profile: default
Enter region: us-east-1
Append to /Users/alex/.pb/siteconfig.yaml? [y/N] y

For Redshift Serverless

Which Redshift Service you want to connect with?
        [c]Redshift Cluster
        [s]Redshift Serverless
s
Enter Redshift Serverless Workgroup Name: serverless-wg
Enter shared_profile: default
Append to /Users/alex/.pb/siteconfig.yaml? [y/N] y

If you choose to use an AWS Secrets Manager ARN to securely retrieve credentials from AWS Programmatic Credentials (option 2c), enter the following details:

For Redshift Cluster

Which Redshift Service you want to connect with?
        [c]Redshift Cluster
        [s]Redshift Serverless
c
Enter Redshift Cluster Identifier: cluster-identifier
Enter secrets_arn: ************************
Append to /Users/alex/.pb/siteconfig.yaml? [y/N] y

A sample connection for a Databricks account is as follows:

Enter Connection Name: test
Enter target:  (default:dev):  # Press enter, leaving it to default
Enter host: a1.8.azuredatabricks.net # The hostname or URL of your Databricks cluster
Enter port: 443 # The port number used for establishing the connection. Usually it is 443 for https connections.
Enter http_endpoint: /sql/1.0/warehouses/919uasdn92h # The path or specific endpoint you wish to connect to.
Enter access_token: <password> # The access token created for authenticating the instance.
Enter user: profiles_test_user # Username of your Databricks account.
Enter schema: rs_profiles # A segregated schema for storing tables/views created by Profiles
Enter catalog: your_rudderstack_db # The database or catalog having data that you’ll be accessing.
Append to /Users/<user_name>/.pb/siteconfig.yaml? [y/N]
y

• Connection Name: Name of the connection in the project file.
• Target: Environment name, such as dev, prod, test, etc. You can specify any target name and create a separate connection for the same.
• Host: Host name or URL of your Databricks cluster.
• Port: Port number for establishing the connection, usually 443 for https connections.
• http_endpoint: Path or specific endpoint you wish to connect to.
• access_token: Access token for authenticating the instance.
• User: Username of your Databricks account.
• Schema: Name of the schema to store your output tables/views.
• Catalog: Name of the database or catalog from where you want to access the data.

RudderStack currently supports Databricks on Azure. To get the Databricks connection details:

1. Log in to your Azure’s Databricks Web UI.
2. Click on SQL Warehouses on the left.
3. Select the warehouse to connect to.
4. Select the Connection Details tab.

A sample connection for a BigQuery account is as follows:

Enter Connection Name: test
Enter target:  (default:dev):  # Press enter, leaving it to default
Enter credentials: json file path: # File path of your BQ JSON file, for example, /Users/alexm/Downloads/big.json. Entering an incorrect path will exit the program.
Enter project_id: profiles121
Enter schema: rs_profiles
Append to /Users/<user_name>/.pb/siteconfig.yaml? [y/N]
y

• user_main_id: Rudder ID generated by Profile Builder. Think of a one-to-many relationship, with one Rudder ID connected to different IDs belonging to same user such as User ID, Anonymous ID, Email, Phone number, etc.
• other_id: ID in input source tables that is stitched to a Rudder ID.
• other_id_type: Type of the other ID to be stitched (User ID, Anonymous ID, Email, etc).
• valid_at: Date at which the corresponding ID value occurred in the source tables. For example, the date at which a customer was first browsing anonymously, or when they logged into the CRM with their email ID, etc.

• user_main_id: Rudder ID generated by Profile Builder.
• valid_at: Date when the feature table entry was created for this record.
• first_seen, last_seen, country, first_name, etc. - All features for which values are computed.