Recently, we shared our updated mission and vision for Meltano, and now I’ll spend some time sharing how we intend to bring them to life. This post is all about our strategy.
Strategy is your competitive game plan. It describes the overarching approach of how you will achieve your objectives. I want to share with you how we think about Meltano and the larger data ecosystem from both the business and product side. After reading this post, our expectation is that you’ll better understand our approach to building in this market, how we think about everyone in the data ecosystem, and how our roadmap flows from this strategy. We will have another post that ties these strategies to specific tactics and a more detailed development roadmap.
Meltano Has Big Ambitions
We aim to become a public company one day. The ethos and ambition of GitLab, which recently went public itself, is in our DNA. We are building a fantastic, open core DataOps data platform infrastructure that organizations large and small will love using and that, eventually, some will happily pay for. And we are building an organization where people love to work and are able to do the best work of their careers.
At the core of this are our values of “believe in better,” “progress over perfection,” “together we thrive,” and “kindness is key.” Values are important because they inform the actions we’ll take to achieve our mission and vision.
The road ahead is long and the future is unknown, but there are several key elements of our strategy that we’ll follow, even if our specific tactics and features change.
Our Next Milestone: DataOps Platform Infrastructure Product-Market Fit
Our mission is to enable everyone to realize the full potential of their data and our vision is to be the infrastructure for every team’s ideal data stack. To this end, we are building a modular, open source DataOps platform infrastructure that will bring the advantages of DevOps and other software development best practices to the entire data lifecycle. The next milestone on the journey to realizing this vision is finding product-market fit (PMF) with this DataOps platform infrastructure vision. PMF means getting Meltano to the point where it is effectively solving an urgent problem and is generating deep value that users would be willing to pay for.
Based on our usage data, the feedback we’ve been getting from our users, and their requests for us to start offering a SaaS edition of Meltano, we have good PMF within data integration, but our ambition extends to the entire data lifecycle. Finding PMF for our DataOps infrastructure platform is necessary for us to earn the right to keep working toward our vision and continue growing as an organization to get there faster. Once we’re starting to see clear signs that users are excited about what we’re doing and seeing real value from our DataOps platform infrastructure as it develops, we’ll be able to double down on our vision and get the resources to keep improving it and bring it to as many people as possible.
We believe the following Strategy Pillars, along with our Business and Product Strategy, will get us there.
Strategy Pillars
The three pillars detailed here are the components of our strategy that we expect to stay consistent no matter the stage of the company. They’re connected to the mission and vision but bring more clarity about what we aim to do.
Continually Prove the Value of Our DataOps Platform Infrastructure
DataOps, casually known as DevOps for Data, is the application of software engineering best practices, such as version control, code review, automated end-to-end testing, and isolated environments to data teams and workflows.
We believe that DataOps will be critical to how organizations build their high-performing data cultures going forward. If you look at Gartner’s Hype Cycle for Data Management, you’ll find that DataOps is rising towards the peak of inflated expectations. Further along the curve you’ll find “Data Engineering” in the trough of disillusionment. Past that you’ll see “Data Integration Tools” nearing the plateau of productivity. I highlight each of these because it ties together our entire strategy.
Bringing DataOps to maturity means we have to iteratively add value at every step of our journey, to increase people’s confidence in the advantages of the DataOps platform infrastructure. Meltano has brought open source and DataOps to data integration, and we will bring it to the rest of the data lifecycle in collaboration with data professionals and the data ecosystem (see the section on Product Strategy below for more details).
Deep Commitment to Open Source
We’re committed to always having a free, open source edition of Meltano. This is a fantastic way for individuals and teams that don’t need enterprise-level features to build and manage their data stacks. GitLab and other companies have shown that the buyer-based open core business model is an effective way for open source projects to support their ongoing development. And they’ve done this in a way that doesn’t conflict with the needs of the broader open source community and user base.
Open source is also critical to maximizing the number of people who can benefit from Meltano and the impact of our work. Our mission includes everyone and our aim is to make it as easy as possible to start using and getting value from Meltano. Additionally, a broader user base will increase the pace of feedback and contributions and lead to a better product for all users.
Prioritizing the Community
Community is a core value for our organization and product. Communities like Singer, dbt, and Locally Optimistic have shown the value of a large number of people in the open sharing of tooling and knowledge. With Singer in particular, we’ve invested heavily to support the community as best we can: we’ve made Meltano the best way for the Singer community to run taps and targets in production, and we’ve built tools and resources to make building, discovering, and maintaining connectors fun and easy.
We will continue investing in the Meltano community to make it a great place to give and get help, connect with other data professionals, and contribute to building the future of data tooling and the data profession.
Business Strategy
Just as we have three strategy pillars, there are three components to our near-term business strategy. We’re privileged to be funded by amazing investors who support our mission and vision and have encouraged us to think about Meltano as a business in this way. We expect this to grow and evolve as we find PMF and have users ready to pay.
Focus on growing the open source community
The success of companies like GitLab, dbt Labs, and Elastic has shown that given a large enough base of users that love and help improve an open source product, a successful business can be built around it, sustained by the revenue of enterprise adoption. Therefore, our focus right now is to grow the usage, lovability, and value created by Meltano, and the open source ecosystems we integrate with, as much as possible. This means solving immediate user needs with the current capabilities of Meltano, while bringing our community along with us as we realize the promise of DataOps and build the future of data tooling together.
A big part of that community are our partners who implement data solutions for their clients. Whether it’s deploying Meltano, building a Singer tap or target, or doing custom data work, these are the people we want to send our community to when they have immediate data needs. We support our partners by directing users their way when it makes sense, and they give back through their feedback, contributions to the product and ecosystem, and growth of Meltano’s deployment base.
Educate Users and the Market about the Advantages of DataOps
The data profession is growing. Roles and specializations are changing, more people want to gain data skills, and more companies realize the need to improve their data strategies. We believe this growth, desire, and need will continue. As more people find their way into the data profession and existing professionals reevaluate their approach, we will teach them about what DataOps in general, and Meltano specifically, can do for them and their organizations.
We do this today with Meltano Academy, community events like Office Hours and Demo Days, and content on our blog. But it’s also essential to do this via the product itself: seeing is believing, and the most effective way to get new people to recognize the value of DataOps is by improving onboarding and the overall user experience. We aim to make DataOps feel easy and Meltano a tool people love to reach for.
Focus on Open Source Features for Self-managed Users
We’re focusing solely on the open source product and community until we reach PMF for our full DataOps platform infrastructure and have dozens of companies asking us for hosting, support, and enterprise SLAs and functionality. Shifting too early to focusing on the needs of the enterprise can be a distraction from our focus on building a great community and product for everyone and alter the incentives for everyone in the ecosystem.
By deeply committing to the Meltano open source product and community first and growing the number of people that love Meltano, organizations that get value from it, and products it integrates with, we believe we’ll be better able to support larger enterprises in the long run.
Product Strategy
With the Strategy Pillars and overall Business Strategy laid out, it’s now time to shift to our specific Product Strategy. The goal here is not to get into specific features, but rather to describe the outlines of the trail ahead. You should be able to look at this and understand how it maps to our near-term roadmap and priorities, and how it will get us closer to PMF. Douwe Maan, Meltano Founder and CEO, briefly touched on aspects of the product strategy in his post on our mission and vision, but I want to dive deeper here.
Start with Open Source Data Integration
As Douwe mentioned in the original pivot post and in the mission and vision post, we have focused on bringing DataOps and open source to data integration for the past year and a half. Data integration is the first part of any data journey and a critical component to “get right.” Before you can provide insights you have to first make sure the right data is in the right place and in the right format. Meltano is a fantastic choice to solve your data extraction, loading, and transformation needs and introduce you to the advantages of DataOps, even if you’re not yet explicitly looking for a DataOps solution.
We leverage the Singer standard and its great community and will continue to invest in the ecosystem through initiatives like the Singer Working Group, MeltanoHub for Singer, and the Meltano SDK for Singer Taps and Targets. Since our goal is to enable teams to build their ideal data stack, we envision Meltano working with other extract and load tools in the future and adding value to them as well.
Build for Self-Managed and SaaS
SaaS will be a very important part of our business once we have product-market fit, and it’s clear that more organizations will move to SaaS tools over the next decade. However, there is still a large need for companies to manage their own data tooling in their own environments due to privacy, security, and compliance needs. Because of this, we aim to make a great product for both sets of users. There should be no SaaS-only features as we believe the self-managed experience should be just as good as SaaS. By focusing on self-managed scenarios for the time being, and keeping our SaaS in mind, we will build an inclusive solution that works for all users.
Enhance Existing Integrations and Continue Earning Users’ Trust That We Can Integrate
Meltano currently adds value to its plugins in three areas. First by bringing in package management functionality that enables easy and automatic installation, configuration, and deployment. Second, by introducing critical DataOps capabilities that enable more effective collaboration, experimentation, and increased confidence in all output. And third, by creating meaningful abstractions and integrations between plugins that make it easy to create cross-plugin workflows with automatic data and configuration exchange. Taken together, this means that adopting Meltano as your DataOps platform infrastructure and the foundation of your modern data stack makes it greater than the sum of its parts.
We’ve shown with Singer that we’re adding value in all three of these areas, but we need to improve in the third area when it comes to our support for dbt and Airflow. As we continue to bring the value of DataOps to more parts of the data stack, we will also improve the integrations between plugins to unlock even more value than each plugin provides alone.
Grow Available Plugins to Enable Users to Build an End-to-end Stack
In support of the larger vision, we’ll continue growing the library of plugins that enable users to build an end-to-end data stack on top of Meltano. This means adding support for alternative orchestration tools, as well as adding new capabilities into Meltano to support other layers of the stack such as visualization, metrics, quality, and observability. We want to be the easiest way to try, set up, and deploy different tools, alone and in combination with others, for all parts of the modern data stack. With each new plugin, users will gain immediate value from the DataOps capabilities of Meltano as well as the deeper integration with other parts of the data stack, until the entire stack lives in one place for the entire team to collaborate on.
Show Vendors How Meltano Helps Them and Their Users
By providing a stable foundation for data teams and focusing on integration between different components, we support vendors of data tools by enabling them to focus on what they do best in their chosen area and leaving the rest to us. Meltano will make it easier than ever for users to try their tools out, integrate them with others, as well as manage installation, configuration, and deployment. We expect that once the value Meltano brings to the wider data tooling ecosystem is clear, new and existing tools will actively want to be supported by Meltano, and we will receive contributions from tool vendors and users to grow the plugin library. Since our vision involves bringing DataOps best practices to your ideal data stack, we do not intend to restrict your options and will also explore how we can best integrate with vendors’ paid and SaaS offerings.
Show How an End-to-end DataOps Platform Infrastructure Opens Up New Opportunities
With a strong base of data integration and support for additional plugins across the data lifecycle, we’ll be able to show how having Meltano as the infrastructure for your modern data stack enables additional emergent value beyond the critical capabilities centralized package and configuration management and DataOps provide. Once Meltano has full visibility into and control over all of your tooling, we’ll be able to provide a centralized control center for all data workflows across an organization. This means we can surface useful insights about your data flows, such as overall costs and quality, but also provide observability, metadata management, and governance capabilities that are so often disconnected from the actual work being done.
Our Next Milestone: DataOps Platform Infrastructure Product-Market Fit (PMF)
As we make progress on each strategic point, we expect to see increases in general usage of Meltano, usage of DataOps-specific features such as environments and end-to-end plugin testing, and usage of plugins for stages of the lifecycle beyond data integration. This data, coupled with community sensing mechanisms such as our public issue tracker, our Slack group, and frequent feedback from our partners and users, will tell us if and how quickly we are progressing towards PMF for Meltano as a DataOps platform infrastructure.
Once we get there, we will work with our partners and potential customers to build a SaaS offering for hosting Meltano-based data stacks. We’ll also work with them to understand what enterprise features they’d expect and would be willing to pay for. If this is of interest to you, we would love to talk to you. We expect to have a paid offering in the latter half of 2022.
A Strategy Based on Values
Our values are crucial to how we work, present ourselves to the world, and interact with our community of users and other players in the ecosystem. These values have guided the strategy presented here.
In the spirit of Transparency, our roadmap and development milestones will continue to be open to everyone. Our roadmap will tell you what we’re aiming to achieve in the next months and quarters, and specific milestones will tell you what’s prioritized for any given week.
I’ll close out this post by repeating what Douwe said in the Mission and Vision post:
The best tools are those built in close collaboration with their users: for data people, by data people. Contributors of all types working together every day in the same code base and issue tracker towards a common goal. Your ideas and unique perspectives are critical to our mission. We cannot build your team’s ideal data tool or support you in building your ideal data stack without your input.
If our mission, vision, strategy, and values resonate with you, we invite you to come make it happen with us. Our roadmap will be updated in December to detail how we’re thinking about development in 2022.
In the meantime, we encourage you to join our Slack community of over 1800 data professionals. Be part of the conversation and be the first to learn about what we’re up to. Also, be sure to check out our job openings and values in case you’d like to work on these solutions full-time!