By Arjun Mehta
DORA Metrics Are Not Enough: What They Miss About Your Product
DORA metrics are fantastic at measuring engineering performance. But they measure how fast you ship, not what you're shipping or whether it matters.
A team with elite DORA metrics (daily deployments, < 1 hour lead time, high reliability) could still be shipping features nobody wants.
This is the hidden danger of DORA metrics: they optimize for speed, but speed toward the wrong destination is worse than being slow.
What DORA Metrics Actually Measure
The four DORA metrics measure:
- Deployment frequency: How often
- Lead time: How fast
- Change failure rate: How reliable
- MTTR: How quickly we recover
All four answer the question: "Can we ship reliably and frequently?"
They don't answer:
- Are we shipping the right things?
- Do customers care about what we're shipping?
- Are we learning from shipping?
- Are we making money?
The Elite Team That Shipped the Wrong Thing
Imagine a team with perfect DORA metrics:
- Deploy 10 times per day
- Lead time < 30 minutes
- 99% success rate
- MTTR < 15 minutes
But they're shipping features nobody wants. Users don't adopt them. Revenue doesn't increase. The product slowly dies.
DORA metrics can't tell you this is happening.
What DORA Metrics Miss
Miss 1: Product-Market Fit
You could have perfect engineering metrics while completely missing the market.
Example: A fintech startup ships 5 new features per week. All reliable, fast, deployed safely. But customers aren't signing up. Product-market fit is terrible.
DORA metrics would say: "Great engineering." Reality: "Wrong product."
Miss 2: Feature Adoption
You could ship a feature perfectly, and nobody uses it.
Example: You ship a new dashboard in 2 days (elite DORA). But 95% of users never open it.
DORA metrics would say: "Shipped fast." Reality: "Wasted engineering time."
Miss 3: Customer Impact
You could improve metrics while making the product worse for customers.
Example: You speed up a feature release from 1 week to 1 day by cutting QA. Now you ship bugs faster. Users get worse experience.
DORA metrics would say: "Faster delivery." Reality: "Worse product."
Miss 4: Learning Velocity
You could be shipping constantly without learning anything.
Example: You ship 20 experiments per quarter without running any A/B tests. You have no idea which ones worked.
DORA metrics would say: "High deployment frequency." Reality: "Shipping without learning."
Miss 5: Technical Debt Trade-offs
You could achieve high DORA metrics by accumulating technical debt.
Example: You ship features fast by skipping refactoring and taking shortcuts. DORA metrics are elite. But in 6 months, feature velocity drops 50% because you're drowning in debt.
DORA metrics don't account for the hidden cost.
The Metrics You Need
Metric 1: Feature Adoption
What it measures: Are users actually using what we shipped?
How to measure:
- % of users who have tried the feature
- % of users who use it regularly
- Time to first use
- Repeat usage rate
Why it matters: A feature nobody uses is wasted engineering time.
Metric 2: Business Impact
What it measures: Does the feature move business metrics?
Examples:
- Revenue increased by $X
- Conversion rate improved by Y%
- Customer acquisition cost decreased by Z%
- Retention improved
Why it matters: Engineering is a cost center unless it drives business results.
Metric 3: Learning Velocity
What it measures: How much are we learning about customers?
Examples:
- Number of experiments run
- User interviews conducted
- Features iterated on based on feedback
- Hypotheses validated or invalidated
Why it matters: Shipping fast in the wrong direction is worse than shipping slow in the right direction.
Metric 4: Quality Metrics
What it measures: Are we maintaining quality while shipping?
Examples:
- Defect escape rate (bugs in production)
- Support tickets from new features
- Incident impact (downtime, affected users)
- Technical debt ratio
Why it matters: Elite DORA metrics + poor quality = false sense of success.
Metric 5: Developer Satisfaction
What it measures: Are engineers happy?
Examples:
- Developer satisfaction scores
- Code review time
- Time in meetings vs time coding
- Onboarding experience
- Retention
Why it matters: Burned out engineers quit. High DORA metrics + high burnout = unsustainable.
Combining Metrics
The right approach is to measure all three dimensions:
Dimension 1: Engineering Excellence (DORA Metrics)
- Deployment frequency
- Lead time
- Change failure rate
- MTTR
Dimension 2: Product Success (Product Metrics)
- Feature adoption
- Business impact
- Learning velocity
- Customer satisfaction
Dimension 3: Team Health (People Metrics)
- Developer satisfaction
- Retention
- Onboarding time
- Career growth
The Danger of Single-Metric Optimization
If you optimize DORA metrics alone, you get:
- Fast shipping of wrong features
- Burned out engineers
- Accumulated technical debt
- Low-quality code
- High churn
This is worse than shipping slow with high quality and learning.
Getting Started
- Track DORA metrics (engineering excellence)
- Add product metrics (are features being adopted?)
- Monitor team health (are people okay?)
- Review all metrics together in retrospectives
- Optimize for the combination, not any single metric
DORA metrics are necessary but not sufficient. Add product and people metrics to get the full picture.
Frequently Asked Questions
Q: Should we deprioritize DORA metrics if product metrics are bad? A: No. You need both. Elite DORA + poor product = wrong features shipped fast. Poor DORA + great product = right features shipped slowly. Optimize for both.
Q: How do we handle trade-offs between DORA and product metrics? A: They usually don't conflict. Taking time to validate features (product metric) doesn't require slow deployments (DORA metric). They're orthogonal.
Q: What if business metrics don't improve despite elite DORA metrics? A: That's a product strategy problem, not an engineering problem. You're shipping the wrong things. Work with product to understand the real problem.