The Meltano team is excited to announce the official launch of MeltanoHub!
MeltanoHub, or the Hub for short, is the best place to discover Singer Taps and Targets and all plugins supported out-of-the-box by Meltano.
There are over 200 taps and targets on MeltanoHub, making it the most comprehensive catalog of open source data connectors anywhere.
Since pivoting to focus on ELT last year, we’ve done many things to lift up the Singer ecosystem. We’ve made Meltano a fantastic way to run taps and targets and deploy them in a production environment. We’ve created the SDK for Taps (and soon for Targets!) to make it as easy as possible to build high quality connectors without needing to understand the full Singer specification (though if you’re interested in the details of the specification, we’ve worked to simplify it so it’s easier to understand everything).
And now we’ve built MeltanoHub to solve one of the last challenges facing the Singer ecosystem: discoverability.
One of the challenges for existing Singer resources, such as the singer-io GitHub namespace or the Singer.io site, is that they aren’t regularly updated and don’t showcase the activity of the many participants in the ecosystem. In reality, there are over 100 existing taps not listed on the canonical Singer repositories and of the ones that are listed, there are more active and updated forks that aren’t surfaced to the community. What this means is that it’s not clear to anyone, let alone participants in the ecosystem, that Singer has the most connectors of any data integration tool (open or closed source)! These sites also don’t take the opportunity to showcase the dozens of active consultancies and companies who build Singer connectors or use them within their own products.
We believe the launch of MeltanoHub for Singer makes these great parts of the Singer ecosystem evident while simultaneously elevating the ecosystem to the next level of usefulness.
We had three main goals with the launch of MeltanoHub:
- Showcase the majority of connectors that already exist within the Singer ecosystem
- Make the directory of taps and targets better than what exists today
- Enable any person or organization to build on top of the large library of Singer connectors
We believe we’ve achieved each of these goals with this launch. By pulling data from GitHub we’ve been able to list 201 taps, but we know there’s more we have yet to add! For each connector we’ve curated the best forks and repositories, added metadata, and stored the information in a YAML file (validated via a JSON Schema) that can be iterated on in collaboration with the community. We’re also pulling metrics from GitHub for every tap so users can get a sense of the quality and activity of a given variant.
All of this work is open to everyone via the website and repository, but also via an API. We have endpoints that return a JSON listing of all taps and targets on the site. Meltano itself will be driven by this data and we hope others use this data to build their own catalog of taps and targets within their products. All of these features are detailed more in the Hub docs and we’ll share in future blog posts more about the engineering behind MeltanoHub.
The Future of Singer is Bright
The Meltano team will continue to lift up the Singer ecosystem through MeltanoHub, the SDK for Taps and Targets, and via Meltano itself. We believe in a truly decentralized open source ecosystem and MeltanoHub reflects that. In the coming months we have many planned iterations for the Hub including features like adding more metrics to each connector page, adding more details about a connector’s capabilities and settings, increased automation around quality metrics, adding more taps as supported out-of-the-box by Meltano, and deeper integration between the SDK and the Hub.
We think you’ll find value in the Hub, but we want to hear from you! If you see anything that can be improved, make an issue in the project or, better yet, make a merge request with the change! Also, join the community on Slack where you can keep up to date on everything Singer and Meltano.