Meltano V1.0 Release & Speedrun Test

This morning, we announced Meltano has reached our v1.0 stable release, and the videos below are great for new and potential contributors looking to see our release process.

Release Process

Meltano’s release process is documented here, and in this video we talk through the steps as well as a tricky issue with version 1.0 (which is actually 1.0.1 in PyPi because of a release back in February that took the 1.0 tag by accident):

Post Release Speedrun

In addition to automated testing, after each release we perform and record a “speedrun” (a term we’ve stolen from expert gamer videos) to make sure the user experience is quick and integrated.

We hope you enjoy v1.0, and that you will check out all the improvements we’ve made to Meltano!

Meltano Graduates to Version 1.0

Today we’re happy to announce Meltano’s version incrementation to 1.0, signifying a point of product maturity that offers a solid end-to-end user experience to extract data and turn it into dashboards.

If you haven’t used Meltano yet you can get started in 60 seconds or less by installing Meltano’s 1-Click App in the DigitalOcean Marketplace and then walk through the Getting Started Guide. When you use our link and create a new DigitalOcean account, you will receive $50 of credit over the first 30 days.

We all encourage all existing Meltano users to upgrade to version 1.0

Meltano began as an internal skunkworks project at GitLab ~18 months ago, and over the past 6 months the core team and contributor community have worked together to reach this milestone. We are grateful for all these contributions, and appreciate our colleagues inside GitLab for their warm support for our unconventional “internal startup” approach (more on this in a future post!).

What It Means to Make It To “V1”

Unlike GitLab, which began with a focus on version control and gradually made the transition to managing the entire DevOps lifecycle, the vision for Meltano has been about simplifying the DataOps workflow from day one. With this big vision, we have grappled with the complexity that comes from making choices on behalf of users and needing to satisfy a range of technical know-how levels from experienced data engineer and scientist to freshly minted analyst.

At the beginning of July, we established key criteria for reaching v1.0 to address these challenges and deliver a working product:

  • UI-based product that is fully usable with the mouse, no command line or coding required (DigitalOcean 1-Click App)
  • End-to-end experience that takes a user from first data extraction to dashboard visualization in minutes (YouTube speedruns)
  • Stable release we can guarantee backward compatibility to and reliable support for (logging, upgrade from the UI, extensively dogfooded and documented with all major bug resolved)
  • Do not invest much more in proprietary Meltano tech for Meltano Model and Meltano Analyze, we should find open source solutions for these phases and integrate them instead

While this might sound like a simple list on its face, the surface area of Meltano as a data pipeline management tool is already large enough that the work required to ensure an end-to-end experience was the bulk of the heavy lifting.

We put the focus on these areas above all else because our core KPI to measure whether Meltano is working is our ability to consistently grow MAUI. With the release of Meltano v1.0 we now offer a product experience worth coming back for and we can’t wait to see what you’ll create!


Stay up to date with Meltano’s progress by subscribing to our newsletter.

Announcing the Meltano 1-Click App, Now Available in the DigitalOcean Marketplace

We are so happy to announce that Meltano is now supported as a one-click installer within the DigitalOcean Marketplace, enabling users to get started with Meltano in minutes with a pre-configured DigitalOcean Droplet in less than 60 seconds.

Install Meltano on DigitalOcean

Ready to take Meltano for a spin? Get started with these simple instructions:

Once you have Meltano up and running, our Getting Started Guide provides a walkthrough of Meltano’s data-to-dashboard capabilities, so you can begin extracting data, building models, and performing analysis in minutes!


We are grateful to our partners at DigitalOcean, Nick Wade, Ryan Quinn and Daniel Levy for helping us get this released. The DigitalOcean Marketplace launched earlier this year, and we are thrilled to join companies like GitLab, Docker, CloudBees, WordPress and many more.

Meltano 0.44 Released

If this is your first time exploring Meltano for your company’s data pipeline management, you can follow our Installation Guide and Getting Started Guide to get going in minutes!

New

  • #950 Removed the Analyze connection configuration: Meltano will now infer connections out of each loader configuration.
  • #1002 Analyze UI now displays the Topic’s (analysis model’s) description text if applicable
  • #1032 Add ‘Model’ and ‘Notebook’ to main navigation to communicate that Meltano plans to empower users with modeling and notebooking functionality
  • #949 Add “Log” button and dedicated sub-UI for tracking an ELT run’s status more granularly
  • #932 Meltano can now be upgraded from the UI directly.

Changes

  • #1045 Make it clear that ‘meltano add’ is not hanging while installing plugins
  • #1000 Update Getting Started guide with updated screenshots and content
  • #854 Charts now use pretty labels rather than the ID
  • #1011 Removed “Catch-up Date” in favor of default “Start Date” of extractor
  • #578 Remove support for tap-zuora.
  • #1002 Update discovery.yml with explicit kind: password metadata (we infer and set input types of password as a safeguard, but the explicit setting is preferred)
  • #1049 Change default target-sqlite database name to warehouse to not conflict with system database
  • #949 Update the way Meltano handles logs for ELT runs: Every elt run is logged in .meltano/run/logs/{job_id}/elt_{timestamp}.log. That allows Meltano to keep logs for multiple, or even concurrent, elt runs with the same job_id.
  • #949 Update “Create Pipeline” redirect logic based on the previous route being ‘transforms’ (this is a UX win setting up the user with the sub-UI for the next logical step vs. requiring a manual “Create” click)
  • #1051 Automatically set SQLALCHEMY_DATABASE_URI config to system database URI

Fixes

  • #1004 Fix error when deselecting last attribute in Analyze
  • #1048 Fix various actions that should have been mutations and did minor code convention cleanup
  • #1063 Fix the “Explore” button link in Dashboards to properly account for the namespace

Breaks

  • #1051 Remove MELTANO_BACKEND e.a. in favor of –uri CLI option and MELTANO_DATABASE_URI env var
  • #1052 Move system database into .meltano directory to indicate it is owned by the app and not supposed to be messed with directly by users

Instructions for upgrading to the most current version of Meltano are available in our documentation.

To see the full history of improvements to Meltano, please review our CHANGELOG

Meltano 0.43 Released

If this is your first time exploring Meltano for your company’s data pipeline management, you can follow our Installation Guide and Getting Started Guide to get going in minutes!

New

  • #1014 Meltano now logs all output from each meltano elt run in a log file that uses the unique job_id of the run. It can be found in .meltano/run/logs/elt_{job_id}.log.
  • #955 Establish baseline for demo day and how they should be run

Changes

  • #891 Contributors can run webapp from root directory

Fixes

  • #1005 Fix installed plugins endpoints listing identically named plugins of different types under wrong type

Instructions for upgrading to the most current version of Meltano are available in our documentation.

To see the full history of improvements to Meltano, please review our CHANGELOG

Meltano 0.42.1 Released

If this is your first time exploring Meltano for your company’s data pipeline management, you can follow our Installation Guide and Getting Started Guide to get going in minutes!

Changes

  • #987 Update routing to match labels (verbs vs. nouns) in effort to subtly reinforce action taking vs. solely “thing” management

Fixes

  • #992 Fix missing GA scripts
  • #989 Fix UI/UX documentation regarding recent removal of view-header
  • #994 Fix stale Pipelines Count in main navigation Pipeline badge
  • #999 Update yarn dependencies to resolve peer dependency warning
  • #1008 Fix error on “Create Pipeline Schedule” modal when no plugins have been installed

Instructions for upgrading to the most current version of Meltano are available in our documentation.

To see the full history of improvements to Meltano, please review our CHANGELOG

Meltano 0.42 Released

If this is your first time exploring Meltano for your company’s data pipeline management, you can follow our Installation Guide and Getting Started Guide to get going in minutes!

New

  • #976 Route changes will update page title in the web app

Changes

Fixes

  • #965 Fix a regression that prevented the Meltano UI to reach the Meltano API when using an external hostname.
  • #986 Fix an issue where the Orchestration page would not show Airflow even when it was installed.
  • #969 Fix an issue where the Meltano Analyze connection would not respect the port configuration.
  • #964 Fix copy button overlap issue with top navigation
  • #970 Fix Meltano’s m5o parser and compiler to properly namespace and isolate the definitions of different custom and packaged Topics.

Instructions for upgrading to the most current version of Meltano are available in our documentation.

To see the full history of improvements to Meltano, please review our CHANGELOG