In office hours this week, we discussed our vision for the upcoming SingerHub as well as several SDK topics such as parent-child streams. Keep scrolling for the recap, or catch up on what you missed using the recording below.
Envisioning a Singer “Hub” for plugin discovery
The most exciting topic for me this week was a resounding confirmation of the many ways SingerHub can improve the process of discovering taps and targets within our ever-growing Singer community. Above and beyond what we have in github.com and singer.io today, we discussed strategic metrics and metadata which SingerHub could expose to users upfront, greatly streamlining the plugin discovery and adoption process. For example, SingerHub could help users evaluate taps based on metrics such as download numbers from PyPi, recency of commits on GitHub, anonymous usage metrics from Meltano, and many others. In addition, SingerHub can generate automated documentation for catalog and config data, which otherwise would have to be discovered by manually scanning GitHub documentation or (gasp!) the plugin code itself. We heard this week a resounding confirmation that community members share our desire to streamline this process and bring rich metadata and metrics together all in one place.
I won’t try to recap everything we discussed here, but please join the discussion using our new #singerhub channel in Slack, and stay tuned for more announcements from Taylor Murphy on SingerHub and how it fits with the Meltano open source vision for DataOps!
New SDK feature proposals
In addition to the Singer Hub vision work mentioned above, we also discussed several SDK related features. Those included a “demo mode” in the SDK, PII tracking, advanced parent-child relationships, and a new idea for explicitly declaring unsupported streams. If your are interested in learning more or contributing to these features, we highly encourage you to join the discussion, either using the links above or by posting your thoughts to our dedicated #sdk channel in Slack.
We hold Office Hours every Wednesday at 9AM Pacific, 16:00 UTC.
Hope to see you soon!
AJ