Meltano 0.20 Released

If this is your first time exploring Meltano for your company’s data pipeline management, you can follow our installation guide and starter tutorial to get going in minutes!

New


Instructions for upgrading to the most current version of Meltano can be found in the version section of our documentation.

To see the full history of improvements to Meltano, please review our CHANGELOG

Meltano Weekly Demo Day: 2019-04-12

In Friday’s Demo Day we show an end-to-end demo using Meltano to run the ETL, pulling data from several SaaS tools, and then showing what is possible with our pre-made “out of the box” transforms, which now ship with Meltano.

We also dug into a conversation exploring whether users will benefit from having separate “projects” or workspaces. This seems like an important thing for collaboration, but is also potentially a feature that is a bit too early in the lifecycle of Meltano given how few users we have.

As always, we welcome your feedback here in the comments as well as in our Meltano Slack community or as an issue in the Meltano project itself.

Meltano 0.19.1 Released

If this is your first time exploring Meltano for your company’s data pipeline management, you can follow our installation guide and starter tutorial to get going in minutes!

New in 0.19.1

Fixes in 0.19.1

  • #576 Fix switching between designs now works
  • #555 Fix meltano discover improperly displaying plugins
  • #530 Fix query generation for star schemas

Instructions for upgrading to the most current version of Meltano can be found in the version section of our documentation.

To see the full history of improvements to Meltano, please review our CHANGELOG

Meltano 0.19 Released

We are thrilled to release so many improvements to Meltano this week, with a special focus on bringing greater feature parity between the Command Line Interface (CLI) and the User Interface (UI). Our vision for Meltano spans many roles in a company, often involving cross-team collaboration of a data engineer and a data analyst. Through these improvements, and many more coming in the next few weeks, we look forward to meeting our users where they’re already getting work done every day.

If this is your first time exploring Meltano, you can follow our installation guide and starter tutorial to get going in minutes!

What’s new in 0.19

Meltano UI

  • Users can now add new projects within the Meltano UI (previously CLI-only)
  • #513 Added initial end-to-end tests for the UI

Extract

  • !544 Add support for extracting data from CSV files by adding tap-csv to Meltano

Transform

Analyze

Orchestrate

  • #514 Add ‘airflow’ orchestrators plugin to enable scheduling

For Contributors:

  • #327 Add meltano add –custom switch to enable integration of custom plugins

Changes in 0.19

  • #455 Updated documentation about target-snowflake

Fixes in 0.19

  • #507 Ensure design name and table name don’t need to match so multiple designs can leverage a single base table
  • #551 Fix HDA queries generated when an attribute is used both as a column and as an aggregate.
  • #559 Add support for running custom transforms for taps without default dbt transforms.

Instructions for upgrading to the most current version of Meltano can be found in the version section of our documentation.

To see the full history of improvements to Meltano, please review our CHANGELOG

Meltano 0.18 Released

This week’s release includes the following changes to Meltano.

New

Changes

  • Remove Snowflake references from advanced tutorial.
  • #2 dbt-tap-zuora Remove custom SFDC related attributes from Zuora Account and Subscription Models
  • Update Contributing – Code Style documentation to including pycache troubleshooting

Fixes

  • #529 Resolve “SFDC Tutorial – ELT Fails due to invalid schema.yml” by #4 dbt-tap-salesforce removing the schema.yml files from the dbt models for tap-salesforce.
  • #502 Fix the situation where an m5o has no joins, the design still will work.

Instructions for upgrading to the most current version of Meltano can be found in the version section of our documentation.

To see the full history of improvements to Meltano, please review our CHANGELOG

Meltano Weekly Demo Day: 2019-03-29

Each week on Friday morning the Meltano team holds our own “Demo Day”, where we share the improvements we’ve made and discuss the benefits to users. Many of these improvements will be further described in our upcoming Monday release, but we also share work in progress that may need a few more sprints to be completed.

In this week’s Demo Day:

  • Micael demos our MVC Airflow integration with Meltano, which will deliver “Solo Piano” orchestration capabilities outlined in our opportunity assessment (#514)
  • Yannis demos how simple it is to connect Meltano to Jupyter Notebooks (#539) + team discussion of personas who will use this and tutorial plans for Python, Scala, and R
  • Jacob demos Meltano UI supporting multiple projects in a single install (crucial for supporting collaboration user stories, as expressed in #516 and #539)
  • Discussion of step-by-step guide (wizard) for improve the new user onboarding experience of the Meltano UI

Check out our milestones to see what we’re working on, and learn about what’s coming next for Meltano. If you have suggestions, please file a new issue. Input welcome!

Meltano 0.17 Released

After last week’s feature rich release, this release was more focused on incremental fixes and improvements while we scope out new MVCs. To see a preview of new features we’re working on, check out Friday’s Meltano Demo Day video.

Several Fixes & Improvements

  • #501 Fix meltano ui crashing when the OS ran out of file watcher.
  • #510 Fix an issue when finding the current Meltano project in a multi-threaded environment.
  • #494 Fix broken links to installation requirements for Starter and Intermediate tutorials + Updated messaging around venv as requirement vs. recommendation
  • #492 A few small contextual additions to help streamline the release process
  • #283 Silence pip’s output when there is no error
  • #468 Added reminder in docs regarding the need for source venv/bin/activate in various situations and added minor copy updates
  • #485 Added various UI unit tests to the Analyze page

Instructions for upgrading to the most current version of Meltano can be found in the version section of our documentation.

To see the full history of improvements to Meltano, please review our CHANGELOG

Meltano Weekly Demo Day: 2019-03-22

Each week on Friday morning the Meltano team holds our own “Demo Day”, where we share the improvements we’ve made and discuss the benefits to users. Many of these improvements will be further described in our upcoming Monday release, but we also share work in progress that may need a few more sprints to be completed.

In this week’s Demo Day:

  • Jacob demos Yannis’ extensive improvements to data for tap-gitlab (#436)
  • Jacob demos MVC for `meltano start` to simplify project creation (#419, #484)
  • Ben demos discovering/adding taps and targets from the UI (#400, #419)
  • Micael demos how users can create/contribute a custom tap (#327)
  • Micael lead conversation around underlying complexity and constraints of the things we’ve chosen to integrate, the value of writing the “glue code” for our customers, and how Meltano is differentiated

Check out our milestones to see what we’re working on, and learn about what’s coming next for Meltano. If you have suggestions, please file a new issue. Input welcome!

Meltano 0.16 released: new CLI commands, improved data and transforms for `tap-gitlab`, Docker image and more

New command line feature: meltano add model [your_model_name]

Before this release, users had to manually copy/paste new models into a project. Meltano 0.16 provides a command line tool to specify which model(s) to load. This enables the use of multiple models on the same data set, and also supports fetching version controlled models for those working in collaboration with others. In 0.16 we also support using the same model across multiple projects, rather than duplicating the same model to each project separately.

We’ve updated our beginner tutorial to demonstrate how to use this command.

Extended GitLab data and default transformations

Previously, tap-gitlab only provided information on GitLab Runner. With today’s release, Meltano now pulls raw data from GitLab’s REST API and offers 3 default transforms:

  • Issues per label
  • Merge requests per label
  • Stats per user

Meltano extracts the following resources, and users can also build their own custom transformations using these building blocks:

  • Branches
  • Commits
  • Issues
  • Projects
  • Project milestones
  • Project Merge Requests
  • Users
  • Groups
  • Group Milestones
  • Group and Project members Tags
  • Releases
  • Group Labels
  • Project Labels

Use Meltano to visualize progress on your team’s GitLab projects and workflow. To get started, check out our quickstart for connecting your GitLab account to Meltano.


And Many More Improvements!

  • Our Docker image now contains everything you need to get started with Meltano
  • Added models to the available plugins in support of meltano add model
  • Default invoice items transform for tap-stripe
  • Many documentation updates including:
    • “Copy to Clipboard” functionality to code block snippets in the documentation
    • Installation and tutorial improvements from our internal dogfooding sessions
    • Troubleshooting buttons to help users add context to a pre-filled bug issue

Instructions for upgrading to the most current version of Meltano can be found in the version section of our documentation.

To see the full history of improvements to Meltano, please review our CHANGELOG