Our first Meltano Demo Day of 2021 was a rousing success and everyone had a great time! If you missed it, the recording of our session is available on the Meltano YouTube channel.
Taylor briefly showed off some updates to the main website including our new Community Code of Conduct and our Product Roadmap. He also shared some data about how active the Meltano Slack group is thanks to all of the wonderful people we have in the community.
AJ gave us an excellent walk-through of the Singer SDK and how to get started building your own tap with the cookiecutter template. Checkout this issue for a list of working samples using the SDK. He also highlighted a VSCode extension he uses called TODO Tree to make it easy to find TODOs scattered throughout your codebase.
Then Julian talked us through his latest contribution in v1.71.0: the ability to include arbitrary Python executables within a Meltano project using the utility plugin type. This led to some good discussion about tools like Yoyo and SQLFluff as well as the future of how to run more complicated data pipelines using Meltano. This led to even more discussion about the future of the commands flag to simplify workflows. Checkout issue #2560 to follow along as that develops.
Lastly, Douwe shared the tap, target, and pipeline he built to solve a personal data problem. Using the Singer SDK, he built a tap to pull data from investing.com. He then built a new target for pushing data into Lunch Money (a tool for managing personal finances). Finally he wired it up in a separate Meltano project and is using GitLab CI to schedule a daily sync of tap-investing to target-lunch-money. It was great to see how Meltano can be used to solve personal data integration challenges and gives a preview of how Meltano may be used to solve for the “Reverse ETL” or “Data Pump” use cases.
Overall, we’re quite happy with how Demo Day went and we look forward to hosting many more of these. We’re always listening to feedback and looking for ways to improve, so please contribute to any issue within the main project or chat with us on Slack if that’s more your style. Thank you to everyone who joined the livestream!
Join the conversation!
Our next Meltano Demo Day is on April 9 and we have our next office hours session on Wednesday, March 31. You can always find a list of all our events in the #events channel on Slack.