Building Meltano in Public: Bimonthly Recap

Last week, it was once again our turn to host a GitLab Group Conversation (a publicly live streamed Q&A on the GitLab Unfiltered YouTube channel) on Meltano!

We used the opportunity to share a recap of:

If you’re curious, check out the presentation on Google Slides and the Q&A on YouTube. The presentation content is also reproduced below, as is an embedded video of the Q&A!

Group Conversation Presentation

2 new team members just joined!

  1. Taylor Murphy joined on March 1
  1. AJ Steers joined on March 8
    • AJ comes from Slalom, where he was a Solution Principal building next-gen products and reusable frameworks to solve today’s hardest data problems
    • As a contributor, he initiated the Singer SDK project: https://gitlab.com/meltano/singer-sdk
    • Announcement blog post: WIP

With Douwe Maan, that makes 3 full-time Meltano team members!

5 releases since the last GC (2021-01-14)

  1. V1.66.0 prevents pipelines from getting stuck in the “running” state forever when their meltano elt process is killed unceremoniously by the operating system or some other mechanism.
  1. V1.67.0 fixes two bugs with meltano schedule run <name>: if the schedule’s meltano elt command fails with a nonzero exit code, it now does as well, and it no longer requires the meltano executable to be in the PATH.
  1. V1.68.0 adds support for entity/attribute selection to tap-gitlab and bumps Airflow to version 1.10.14.
  1. V1.69.0 adds out-of-the-box support for the Quickbooks source and adds support for Airflow 2.
  1. V1.70.0 prevents two pipelines with the same Job ID from running at the same time, adds out-of-the-box support for the Chargebee and Sage Intacct sources, and fixes a bug that prevented Meltano Analyze from working with custom plugins.

17 recent contributions by 9 community members

Done

  1. target-snowflake: Dynamic Precision Fix by Bryan Wise (Halosight)
  2. tap-gitlab:  Implement discovery mode and stream selection by Charles Julian Knight (FIXD)
  3. Add support for Airflow 2.0″ by Michel Radosavljevic (Insurello)
  4. Add `hotgluexyz` variant of `tap-quickbooks` by Hassan Syyid (Hotglue)
  5. Document that tap-postgres depends on libpq by Derek Visch (Auto IDM)
  6. Docs Development Steps NPM run serve:docs Missing Script by Derek Visch (Auto IDM)
  7. Add `hotgluexyz` variant of `tap-chargebee` by Hassan Syyid (Hotglue)
  8. Fix Meltano Analyze with custom extractor by Al Whatmough
  9. Add `hotgluexyz` variant of `tap-intacct` by Hassan Syyid (Hotglue)
  10.  Refactor Job sync implementation by Charles Julian Knight (FIXD)
  11. files-airflow:  Add pipeline details into tags by Pankaj Saini (Let’s Do This)

In development

  1. Support for a generic plugin type for arbitrary python cli plugins by Charles Julian Knight (FIXD)
  2. Add support for shortcut commands for plugins by Charles Julian Knight (FIXD)
  3. Adding pipelinewise target-redshift by Ingo Klose (b.telligent)
  4. Disallow parallel pipelines by default by Charles Julian Knight (FIXD)
  5. Add tap-zendesk variant by Benjamin Maquet (Twilio)
  6. singer-sdk: Fix reading catalog from JSON file by Edgar R. Mondragón (SpotOn)

Other exciting recent and ongoing developments

Recent and projected weekly Slack activity

Join 850+ on Slack!

This week’s priorities

Milestone issue board

Upcoming priorities

Milestones issue board

Epics:

Group Conversation Q&A

Intrigued?

You haven’t seen nothing yet!