Version:

Profiles 0.4.x Changelog

Changelog for Profiles v0.4.x.

  •   2 minute read  

Version 0.4.0

2 March 2023

We are proud to announce the latest version of PB 0.4.0, which includes several new features and improvements.

What’s New

  • Redshift - We are excited to share that we now offer Redshift integration. With YAML, you can now effortlessly create ID stitched and feature table models on your Redshift warehouse, without any difficulty.

  • Incremental ID stitching: You can now stitch together data from multiple sources in incremental mode. When new data is added to your source tables, only that data will be fetched, without needing to reload the whole table each time! This shall result in significant performance improvements from earlier versions, especially if the delta of new data is much smaller compared to what’s already been stitched.

  • Insert: A new command, allowing users to add sample data to their warehouse, without having to manually add them.

  • Schema has been migrated to 25. This includes the following changes:

    • models/profiles.yaml - Renamed entityvar to entity_var and inputvar to input_var
    • pb_project.yaml - Renamed profile to connection.
    • siteconfig.yaml - Renamed profiles to connections.

Be sure to use the migrate auto command to upgrade your project and the connections file.

Improvements

  • The command init profile has been renamed to init connection.
  • Lots of modifications under the hood.

Bug Fixes

Resolved issue on default values in an entity var, ensuring that the values are properly set.

Known Issues

  • Warning: While the run command is being executed, canceling it by pressing Ctrl+C doesn’t work as expected. Though it will stop the program’s execution on the CLI, the query will keep running on the data warehouse. This is a documented Snowflake behavior.
  • In a model, an input can’t use columns named “MAIN_ID”, “OTHER_ID”, “OTHER_ID_TYPE”, or “VALID_AT” in its ID SQL.
  • When creating a connection via init command, pressing the Ctrl+C command doesn’t exit the application.
  • migrate auto jumbles up the order and removes comments.
  • On Redshift, some experimental commands such as discover do not work.
  • The command migrate auto migrates siteconfig in your home directory but not any local one.
  • While working with same type of data in Snowflake and Redshift you might encounter errors where it works on Snowflake but not on Redshift. This is due to the fact that implicit casting of different data types for different function or operator might not be supported on one data warehouse while supported on other.

Questions? Contact us by email or on Slack