You are viewing documentation for an older version.

Click here to view the latest documentation.

Version 0.5.2

5 May 2023

Our latest release offers significant performance improvements, enhancements, and bug fixes to provide a better experience.

What’s New:

• A new command, pb show models, which displays various models and their specifications in the project.
• Ability to exit the application while the run command is being executed.
• Project schema version has been migrated to 30.

Improvements:

• Major performance improvements for Redshift. In large data sets, it will reduce the time taken to create ID stitcher table to less than 1/4th of the time taken earlier.
• insert command now picks the connection specified in the current project folder. If not available, it picks “test” in the connection file.
• Siteconfig is now validated when project is loaded.
• The cleanup materials command now removes SQL models as well.

Bug Fixes:

• Resolved the problem where values with null timestamps were excluded from incremental ID stitcher.
• The insert command was showing a success message even if no tables were inserted in the warehouse. This has been fixed.

Version 0.5.1

11 April 2023

What’s New

• Updated schema to version 28 in the project file.

Improvements

• Changed project path parameter from -w to -p for improved usability.

Bug Fixes

• Addressed a few reported bugs for an improved user experience.
• Implemented performance enhancements to optimize overall system performance.

Version 0.5.0

28 March 2023

This release offers significant new additions and improvements, as well as bug fixes.

What’s New

• Cleanup materials - You can now use the command pb cleanup materials to delete materials in the warehouse automatically, without the need for manual deletion. Just specify the retention time period in the number of days (default 180) and all tables/views created prior to that date will be deleted.

• Schema has been migrated to 27. This includes the following changes:

  • pb_project.yaml - The schema version has been updated from 25 → 27. Also, main_id is removed from id_types as main_id_type is now optional, rudder_id is the main_id_type by default.
  • models/profiles.yaml - To explicitly declare edge source ids, each value in edge_sources now requires a from: key to be appended. Also, if you didn’t define main_id in the project file, then no need to specify here.

Improvements

• In the backend code we’ve enabled registry migration which flattens the registry, enabling incremental ID stitcher to operate on incomplete materials. It also introduces a mechanism for migrating common tables.
• We have implemented better error handling for cases where an incorrect model name is passed. Any errors related to incorrect model names are now properly identified and handled by the system.
• Based on feedback from our users, we have renamed default models from domain_profile<> to user_profile<>.

Due to changes in registry, we will be depricating older versions of PB.

Bug Fixes

• Fixed the bug where some experimental features, such as Discover, were not working for Redshift.
• Addressed the problem where validation errors were incorrectly being triggered when a connection had multiple targets, one of which was invalid. The system now only generates an error if the warehouse target that is being passed has errors.
• In addition to previous one, a few more bugs were fixed that were related to validation.
• Errors were coming for users who had initialized the GIT repository but had not added the remote origin. This issue has now been fixed.

Known Issues

• Warning: While the run command is being executed, canceling it by pressing Ctrl+C doesn’t work as expected. Though it will stop the program’s execution on the CLI, the query will keep running on the data warehouse. This is a documented Snowflake behavior.
• In a model, an input can’t use columns named “MAIN_ID”, “OTHER_ID”, “OTHER_ID_TYPE”, or “VALID_AT” in its ID SQL.
• When creating a connection via init command, pressing the Ctrl+C command doesn’t exit the application.
• migrate auto jumbles up the order and removes comments.
• On Redshift, validate access passes all tests, but run command sometimes fail giving error “permission denied for language plpythonu”.
• Some commands such as insert do not work on Redshift.
• For a few clusters, cross DB references can fail on Redshift.
• The command migrate auto migrates siteconfig in your home directory but not any local one.
• While working with same type of data in Snowflake and Redshift you might encounter errors where it works on Snowflake but not on Redshift. This is due to the fact that implicit casting of different data types for different function or operator might not be supported on one data warehouse while supported on other.