If you’re working with dbt, you need to check these five packages out. They supercharge the dbt workflows of the meltano community members.
Over 150 dbt packages are already on the dbt hub adding functionality to model data, test it, assert its quality, and more!
As dbt has matured, the packages are becoming better and there are some that will save you considerable time and supercharge your workflow.
The Meltano community is full of ELT & dbt experts who actively contribute dbt packages, so we asked them for their favorite dbt packages!
Thanks to all the contributing Meltano community members in our Slack channels. Here are the five top packages and tools that we’ve fallen in love with!
(1) dbt-osmosis enhances your dbt developer experience.
If you work with dbt models, you need to have a way to manage your YAMLs, compile Jinja code to actual SQL, and diff your models!
Even if you’re just starting out you will quickly find yourself handling naming conventions and copying model setups from folder to folder.
dbt-osmosis is a tool that makes all of this way easier. It provides you with a workbench, a GUI you can use to paste your model code, and compile it to SQL. It provides diffs using the compiled SQL, it enables you to define naming conventions, and even enables real-time querying using Jinja code.
Learning dbt-osmosis is super easy as it comes with a dbt-jaffle-shop version to explore all of its features.
(2) dbt-artifacts generates models & views describing your project.
Once you use dbt regularly, you will likely end up wanting to analyze dbt, using dbt. You will want to know how long the model runs take, or how many snapshots you have running overall and many more questions like this.
dbt-artifacts is a package that provides you with pre modeled dbt data. You can use this data inside reports & dashboards, or trigger other workflows based on it.
You get data on every dbt feature resulting in fact tables for snapshot, test, seed, and model invocations as well as individual dimensions providing an overview of all existing (seed) models, tests and snapshots.
The package can be used whether your dbt project runs on dbt Cloud or elsewhere.
(3) dbt-expectations provides a shared standard for data quality inside dbt, not outside.
On mature projects, everyone ends up creating some form of test for the data, not just with the default dbt tests, but tests like “does this column exist”, “column value should be between 0 and 10000”, or even “column value should match this regex expression”.
dbt-expectations solves this need by implementing great_expectations-like tests right into your dbt project.
You have the ability to test for table shape, missing values, ranges, sets, or even combine multiple columns in a single test.
Once you go beyond your first few iterations on a dbt project, in particular one involving a lot of extracting & loading, I suggest you take a look into this package before moving on to larger and heavier “data quality” solutions.
Note: dbt-expectations automatically loads dbt-utils, so you’re kind of getting 2-1 if you decide to go for this package.
(4) dbt-codegen speeds up new projects.
When setting up a new dbt source, have you ever wondered why you need to manually type in column names that are inside a database?
Enough people did, and dbt-codegen was born. It is a package that removes a lot of these manual efforts with a simple CLI call:
Using this package you can automate a lot of tasks you encounter when you’re working with multiple dbt projects, or with quickly growing projects.
These tasks include: generating sources from a database table, generating base models from a source table, generating import CTEs and many more repetitive dbt tasks.
(5) dbt-utils supercharge your tests, and all your default functionality.
Without a doubt the most successful and widely known package in this list is dbt-utils.
It is the swiss army knife of dbt packages and should be explored right after learning your first dbt modeling skills.
Fun features include simple tests like testing for recency, generating GROUP BY SQL statements, and fun Jinja helpers to “prettify”a date string.
Bonus for VS Code users: dbt-power-user, this VS Code extension makes your dbt workflow fantastic.
If you’re using VS Code to develop your dbt models, you will love this bonus tip! The VS Code extension dbt-power-user provides a lot of functionality not available in the standard dbt CLI.
Using dbt-power-user you can generate models from the source definition, execute tests while developing, follow references across your dbt models, and run a model with a simple “play” button.
This extension is in heavy use by the Meltano community.
Whether you’re just getting started or are already an experienced dbt developer, these packages will bring joy into your workday.
If you don’t yet know how to handle packages, make sure to read the getdbt package docs and visit the getdbt package hub.
Want to share your favorite dbt tools & packages? We’re always up for a chat, reach out via slack, Twitter or LinkedIn!