Building Meltano in Public: 6-weekly recap

Earlier this week, it was my turn to host a GitLab Group Conversation (a publicly live streamed Q&A on the GitLab Unfiltered YouTube channel) on Meltano. Just like last time, I used the opportunity to provide a recap of what’s happened in the community and the progress that’s been made since the last update 6 weeks ago.

If you’re curious, check out the presentation on Google Slides and the Q&A on YouTube. The presentation content is also reproduced below, as is an embedded video of the Q&A!

Group Conversation Presentation

12 releases since the last GC (2020-06-09), with 72 changelog items!

  1. V1.35.0 (10 changelog items) makes it easier to manage and version control a Meltano project’s configuration by having meltano config store values in meltano.yml by default, instead of the system database.
  2. V1.35.1 (3 changelog items) fixes 2 minor bugs and makes the output of meltano add and meltano install consistent in how they refer to plugin types and names.
  3. V1.36.0 (7 changelog items) introduces “file bundle” plugins that are used to automatically add plugin-specific files to your Meltano project when adding a plugin like the dbt transformer or Airflow orchestrator.
  4. V1.36.1 (5 changelog items) adds the ability to specify Singer tap stream and property metadata as if it were any other plugin configuration.
  5. V1.37.0 (4 changelog items) lets you instantly containerize your Meltano project using meltano add files docker.
  6. V1.38.0 (5 changelog items) helps you debug your plugin configuration by having meltano config <plugin> list show the current values of settings and where they come from, alongside the setting names and their environment variables.
  7. V1.38.1 (8 changelog items) contains a significant 61-file refactor of how project settings and environment variables are managed internally.
  8. V1.39.0 (6 changelog items) enables more streamlined and consistent configuration of all Meltano and Meltano UI settings using meltano config and the same highly flexible layered configuration model already used to manage plugin settings.
  9. V1.39.1 (9 changelog items) adds various new settings that make it easier to customize Meltano UI and configure it for production.
  10. V1.40.0 (1 changelog item) lets you customize how your custom extractors show up in Meltano UI using a new logo_url property and existing label and description properties. 
  11. V1.40.1 (9 changelog items) makes meltano config and the UI automatically store configuration in the most appropriate location, taking into account whether the project is deployed as read-only, and whether the setting is sensitive or environment-specific. 
  12. V1.41.0 (6 changelog items) adds a --full-refresh flag to meltano elt to perform a full refresh, ignoring any state left behind by previous runs with the same job ID.

Slack activity over the last 30 days

Join us on Slack!

Slack activity since May 1

I’m not sure why we’re seeing a downward trend in weekly active members over the last week, but if that hasn’t turned around by the next GC, I promise I’ll have more to say about it then!

Other things that have happened since the last GC (2020-06-09)

Current milestone

This week, I’ll be working on these issues filed or requested by the community:

RFA is expected to work on:

Group Conversation Q&A