Meltano

Meltano 0.29 Released

If this is your first time exploring Meltano for your company’s data pipeline management, you can follow our Installation Guide and Getting Started Guideto get going in minutes!

Upgrading Meltano

To upgrade Meltano run the following command:

pip install --upgrade meltano

New

  • #724 Add the model-gitlab-ultimate plugin to Meltano. It includes .m5o files for analyzing data available for Gitlab Ultimate or Gitlab.com Gold accounts (e.g. Epics, Epic Issues, etc) fetched using the Gitlab API. Repository used: https://gitlab.com/meltano/model-gitlab-ultimate
  • #723 Add proper signage and dedicated sub-navigation area in views/pages. Standardized the view -> sub-view markup relationships for consistent layout. Directory refactoring for improved organization.
  • #612 Move the plugins’ configuration to the database, enabling configuration from the UI

Changes

  • #636 Refactored connector logo related logic into a ConnectorLogo component for code cleanliness, reusability, and standardization
  • #728 Change error notification button link to open the bugs issue template

Fixes

  • #718 Fix dynamically disabled transforms always running. Transforms can now be dynamically disabled inside a dbt package and Meltano will respect that. It will also respect you and your time.
  • #684 Enables WAL on SQLite to handle concurrent processes gracefully
  • #732 Fix plugin installation progress bar that wasn’t updating upon installation completion

Instructions for upgrading to the most current version of Meltano are available in our documentation.

To see the full history of improvements to Meltano, please review our CHANGELOG

Meltano 0.28 Released

If this is your first time exploring Meltano for your company’s data pipeline management, you can follow our Installation Guide and Getting Started Guideto get going in minutes!

Upgrading Meltano

To upgrade Meltano run the following command:

pip install --upgrade meltano

New

  • !683 Add --start-date to meltano schedule to give the control over the catch up logic to the users
  • #651 Added model installation in the Analyze UI to bypass an otherwise “back to the CLI step”
  • #676 Add pipeline schedule UI for viewing and saving pipeline schedules for downstream use by Airflow/Orchestration

Changes

  • #708 Enable tap-gitlab to run using Gitlab Ultimate and Gitlab.com Gold accounts and extract Epics and Epic Issues.
  • #717 Enable dbt-tap-gitlab to run using Gitlab Ultimate and Gitlab.com Gold accounts and generate transformed tables that depend on Epics and Epic Issues.

Fixes

  • #716 Fix entities UI so only installed extractors can edit selections
  • #715 Remove reimport of Bulma in /orchestration route to fix borked styling

Instructions for upgrading to the most current version of Meltano are available in our documentation.

To see the full history of improvements to Meltano, please review our CHANGELOG

Meltano Dogfooding 2019-06-11

Each week we release on Monday, and then I update my version locally and try to run through out Getting Started Guide. I’ve decided to start doing this in the open, in the spirit of building in the open with full transparency to our process.

Dogfooding in this way is pretty humbling, and as we build out in the open I want to be able to share with our community that I am running into the same issues they are seeing and we are diligently working to fix the bugs.

I’ve embedded the dogfooding video below, and you can read the issue I created (which you can see me write up live in the video) here: https://gitlab.com/meltano/meltano/issues/699

If you have feedback for us on Meltano, you are welcome to email hello@meltano.com or even create an issue on the Meltano repo.

Meltano 0.27 Released

If this is your first time exploring Meltano for your company’s data pipeline management, you can follow our Installation Guide and Getting Started Guideto get going in minutes!

Upgrading Meltano

To upgrade Meltano run the following command:

pip install --upgrade meltano

New

  • !640 Google Analytics logo addition for recent tap-google-analytics Extractor addition
  • #671 Add the tap-google-analytics transform to Meltano. It is using the dbt package defined in https://gitlab.com/meltano/dbt-tap-google-analytics
  • #672 Add the model-google-analytics plugin to Meltano. It includes .m5o files for analyzing data fetched from the Google Analytics Reporting API. Repository used: https://gitlab.com/meltano/model-google-analytics
  • #687 Implemented a killswitch to prevent undefined behaviors when a Meltano project is not compatible with the installed meltano version

Fixes

  • #661 Fixed empty UI for extractors that lack configuration settings by providing feedback message with actionable next steps
  • #663 Fixed Airflow error when advancing to Orchestration step after installing and saving a Loader configuration
  • #254 Fixed meltano init not working on terminal with cp1252 encoding
  • #254 Fixed meltano add/install crashing on Windows
  • #664 Minor CSS fix ensuring Airflow UI height is usable (side-effect of recent reparenting)
  • #679 Fix an issue with meltano select emitting duplicate properties when the property used the anyOf type
  • #650 Add MELTANO_DISABLE_TRACKING environment variable to disable all tracking
  • #670 Update tests to not send tracking events

Instructions for upgrading to the most current version of Meltano are available in our documentation.

To see the full history of improvements to Meltano, please review our CHANGELOG

First Looker and Tableau, Which Data Visualization Companies Could Be Acquired Next?

In a past life, as CEO of Mattermark, I spent a lot of time building lists like this with our customers — investors, M&A deal-makers, and other professionals looking to understand a space. On the heels of Google acquiring Looker last week and Salesforce announcing their acquisition of Tableau today, I’m looking through my competitive landscape with fresh eyes to contemplate potential deals on the horizon.

There are many reasons for M&A, but for the purpose of this post I’ll focus on situations where companies are not (at least visibly) in distress. The deals above combine the two things established companies want most 1) high quality (e.g. locked in, high $ per customer) revenue to fuel further growth and stock price appreciation 2) differentiating technology innovations that can be bought and leveraged faster than they can be build in house.

We’ve also included a long list of acquired companies in this space, which we believe has been quietly consolidating for year past year or two.

This is a collaborative effort, and feedback is welcome. If you’d like to contribute, please leave your comments here or tweet to us @meltanodata.

Press inquiries regarding this story can be directed to press@gitlab.com

Salesforce is Acquiring Tableau for $15.7B

It seems the consolidation train is chugging along nicely. This morning, it was announced that Salesforce* is acquiring business intelligence and data analytics company Tableau for $15.7B, a healthy premium on the company’s $10.8B market cap. While this is an impressive outcome for the 16 year old company, it will be interesting to see what it could mean for their customers and the broader market. Some open questions this morning:

Last week, we explored what Looker’s acquisition by Google could mean for the free open source data analytics movement and ultimately kicked off a new open source project with collaborators at Rakam to develop a spec defining an alternative to LookML.

Meltano is still in the early days, and seeing major transactions like these make us very bullish that this is a space where there is a great deal of untapped opportunity in terms of product offering, business mode, and for us — acquiring a unique customer base. Because we are able to offer a free open source alternative, much smaller companies can rely on Meltano for their business data dashboards without locking themselves into expensive contracts. As consolidation continues, it will be interesting to see how the “low end” disruptive part of the market we are serving evolves.

Interested in getting more involved with Meltano’s mission to bring data analytics tools to everyone? Feel free to leave a comment on this post, Tweet to us @meltanodata or jump into our public Slack channel. You can also reach the whole team anytime with an email to hello@meltano.com

*Disclosure: Meltano’s GM, Danielle Morrill, is a shareholder in Salesforce.

Rakam is Collaborating with Meltano to Define an Open Source Alternative to LookML

Following this morning’s announcement that Google is acquiring Looker, there has been quite a bit of discussion about the future of the open source data analytics space. Seth Rosen at Hashpath writes:

Meanwhile, in the open source data analytics space, there is a similar technological consolidation and integration happening. Specifically, the Meltano project out of GitLab is stitching together fragmented open-source data analytics projects into a single end-to-end platform. Meltano describes itself as “an open source convention-over-configuration product for the whole data lifecycle, all the way from loading data to analyzing it.” Based on the success that GitLab has had with other projects, we predict that Meltano could eventually give the big, proprietary platforms a run for their money.

Seth Rose, HashPath

LookML is a significant part of why Looker was acquired, and we believe there is a path forward to build an open source alternative that helps users define re-usable business logic without having to know how to write complex queries. While Looker is proprietary, the idea of having a portable way to describe the data needed to arrive at a particular insight or dashboard is a general problem. With Looker now part of GCP, this is the right time to work together on an open standard used by multiple companies.

Thank you to Burak and Ilker from Rakam for joining us on our open Zoom call to discuss a path forward today. We connected following the discussion of Looker’s acquisition of Google this morning, and invite anyone else who would like to participate to join us!

The new project can be found at https://gitlab.com/meltano/model-specs

What We Want to Do

Fundamentally, this project needs to deliver on three key steps:

  • Define a Model
  • Compile the Model
  • Generate SQL
YOLO MICAEL

Next Steps

In our open call today, we defined some next steps:

  • Define a new specification for representing data models
  • Define what data models encompass in this context
  • Think through how to describe core concepts like aggregates, dimensions, measurements, metrics, et al.

We invite those in the data modeling space to help, so that everyone can contribute on models rather than creating their own isolated solutions.

Join the project here: https://gitlab.com/meltano/model-specs

About Rakam

Rakam is a product analytics tool that lets companies analyze their customer event data coming from different sources such as Android, iOS and Web. We help companies to create their summary tables with DBT (the event data volume can be up to hundreds of billions!) and analyze their user behavior with features such as funnel, retention, and segmentation in a similar way to Looker.


LIVE: Join Us to Discuss Building an Open Source Alternative to LookML

This morning it was announced that Looker has been acquired by Google for $2.6B. This kicked off a discussion on Hacker News around open source alternatives. We are grateful to Burak Emre Kabakcı of Rakam.io for engaging with us to explore this more deeply in an open Zoom call, and we invite you to join us as we leave the channel open throughout the day to connect with other potential collaborators.

[CALL HAS NOW ENDED]

Check out the results of our conversation and plans for next steps in collaboration with the team at Rakam.