Following this morning’s announcement that Google is acquiring Looker, there has been quite a bit of discussion about the future of the open source data analytics space. Seth Rosen at Hashpath writes:
Meanwhile, in the open source data analytics space, there is a similar technological consolidation and integration happening. Specifically, the Meltano project out of GitLab is stitching together fragmented open-source data analytics projects into a single end-to-end platform. Meltano describes itself as “an open source convention-over-configuration product for the whole data lifecycle, all the way from loading data to analyzing it.” Based on the success that GitLab has had with other projects, we predict that Meltano could eventually give the big, proprietary platforms a run for their money.Seth Rose, HashPath
LookML is a significant part of why Looker was acquired, and we believe there is a path forward to build an open source alternative that helps users define re-usable business logic without having to know how to write complex queries. While Looker is proprietary, the idea of having a portable way to describe the data needed to arrive at a particular insight or dashboard is a general problem. With Looker now part of GCP, this is the right time to work together on an open standard used by multiple companies.
Thank you to Burak and Ilker from Rakam for joining us on our open Zoom call to discuss a path forward today. We connected following the discussion of Looker’s acquisition of Google this morning, and invite anyone else who would like to participate to join us!
The new project can be found at https://gitlab.com/meltano/model-specs
What We Want to Do
Fundamentally, this project needs to deliver on three key steps:
- Define a Model
- Compile the Model
- Generate SQL
In our open call today, we defined some next steps:
- Define a new specification for representing data models
- Define what data models encompass in this context
- Think through how to describe core concepts like aggregates, dimensions, measurements, metrics, et al.
We invite those in the data modeling space to help, so that everyone can contribute on models rather than creating their own isolated solutions.
Join the project here: https://gitlab.com/meltano/model-specs
Rakam is a product analytics tool that lets companies analyze their customer event data coming from different sources such as Android, iOS and Web. We help companies to create their summary tables with DBT (the event data volume can be up to hundreds of billions!) and analyze their user behavior with features such as funnel, retention, and segmentation in a similar way to Looker.