On October 16, I had the honor to host a talk and Q&A on open source EL(T) with Meltano and Singer at a Data Nerd Herd event organized by Ternary Data‘s Joe Reis.
If you’d like to learn why GitLab is so excited about open source data integration, how Singer provides the foundation, and how Meltano completes the picture, watch the recorded talk below, or check out the slides!
Meltano is an open source, self-hosted, CLI-first ELT platform that brings DevOps best practices such as version control, code review, and continuous integration and deployment (CI/CD) to a space historically dominated by proprietary, hosted (SaaS), UI-first solutions, that end up being poorly customizable black boxes in otherwise transparent data stacks.
At GitLab, where Meltano was founded in 2018, we think that the power of data integration should be available to all, and are on a mission to build a truly competitive alternative to existing proprietary EL(T) solutions, in terms of ease of use, reliability, and quantity and quality of supported data sources.
Meltano leverages the existing ecosystem of open source Singer taps and targets for extraction and loading, and supports dbt transformations and Airflow orchestration through a thin plugin-based integration layer. This makes Meltano the easiest way to get started with open source ELT, without imposing arbitrary limitations on those advanced users who want to take a look behind the curtain and interact with the plugins directly.
About the presenter
Douwe Maan leads the Meltano project at GitLab, which he joined in 2015 as its fifth software engineer, when GitLab’s community of open source contributors still greatly outnumbered the in-house engineering team. He became GitLab’s first Engineering Lead, and ran the team responsible for all version control and code review functionality until 2019, when he moved to Meltano to attempt to replicate GitLab’s success in commoditizing DevOps tooling in the data integration space.