A pipeline fails.
Sales opened yesterday’s dashboard before the morning meeting and the numbers haven’t moved. Finance delays a report. Customer Success starts questioning whether they can trust the data. Meanwhile, the only person who knows what’s actually broken is halfway through another piece of work.
They didn’t write the transformation logic. They’re not reading logs over coffee. All they know is: it’s broken and it needs to work again.
Why Pipeline Failures Take Longer Than They Should
Data pipelines fail for mundane reasons most of the time: a credential expires, a source system changes a column name, a job runs long and times out. None of these are exotic engineering problems.
But the evidence of what happened lives in a format built for deep infrastructure work: raw logs, stack traces, git diffs, run history tables. Reading that evidence and turning it into “here’s what broke and here’s the fix” requires a specific, learned skill, and it’s not one most people touching the data stack have day to day.
That gap between “something is broken” and “I understand what and why” is the actual problem, not the failure itself. Every minute spent figuring out why a pipeline failed is a minute the business is making decisions on stale data. The longer a pipeline stays unexplained, the more confidence disappears, not just in the pipeline, but in the data behind it. That cost shows up two ways: escalation, where someone has to stop and dig up the answer for someone else, and confidence, where every unexplained failure chips away at how much the business trusts its own data.
Diagnose is designed for analytics engineers, the people who write the SQL and own the models, but aren’t the deep infrastructure engineers who created the pipeline orchestration or the connectors underneath it. Most analytics engineers spend more time in SQL than Kubernetes. Yet when a pipeline fails, they’re suddenly expected to understand both.
The data stack changes constantly: a connector gets bumped, a schema drifts upstream, a dbt model someone else touched shifts what a table looks like. Analytics engineers are expected to keep up with all of it just to understand why something broke.
That’s where Meltano’s platform advantage comes in. Change control and code are native to the platform, out of the box. If a pipeline runs dbt through Meltano, the platform already knows what changed, when, and by whom, all in one place instead of scattered across a separate git tool and orchestrator. That’s what makes an accurate diagnosis possible in the first place.
What: A Diagnose Button That Explains Failures in Plain English
Runbooks and documentation help with failures someone has already seen before. They don’t help with the failure happening right now, in a particular pipeline, with contextual logs. So instead of writing more docs, we developed something that looks at this failure, this run history, this config, and explains it the way a teammate who already knows the whole stack would.
When a pipeline run fails, a small sparkle-icon Diagnose button appears right next to the “Error” badge on that pipeline’s card. You won’t see it on healthy runs, only failed ones.
Diagnose automatically:
- Gathers everything needed to investigate the failure, including dbt model changes if the pipeline runs dbt through the platform
- Removes sensitive credentials before anything leaves your environment
- Analyses the failure using Claude and your own Anthropic API key
- Returns a clear explanation and a recommended next step
The result shows up in a simple popup: a title, a summary of the failure, the likely cause, and a concrete next step. From there, you can copy the explanation, close it and move on, or hit Copy as Prompt to hand the fix straight to your AI coding tool of choice.
Benefits: Fix Faster, Escalate Less, Trust the Data More
Fix Faster: Get to the root cause in minutes instead of hopping between logs, git history, and dbt changes.
Escalate Less: Resolve most failures yourself instead of routing every red pipeline to whoever created the underlying infrastructure.
Restore Confidence: Restore confidence before stale data turns into bad decisions.
Stay in Control: Diagnose runs on your own Anthropic API key, set up once under Settings > Credentials > Claude, masked on save and never readable back. No key connected, no silent failure, Meltano routes you straight to settings. Repeat clicks on the same failed run reuse the cached diagnosis instead of calling the API again.