Every AI vendor has an ROI number. "300% return in year one." "10x productivity gains." "Millions in savings." The numbers are big, round, and almost always wrong. Not because the vendors are lying, but because the methodology behind most AI ROI calculations wouldn't pass a first-year statistics course. Vincent and I have been pulling apart these claims for clients, and the gap between projected and actual returns follows predictable patterns.
The Short Version
- Most AI ROI projections use baseline assumptions that inflate the result before the model even runs
- The typical enterprise AI initiative delivers 40-60% of projected ROI in year one, not because AI underperforms, but because the projection methodology is flawed
- Confidence intervals on ROI projections are almost never reported, which means the "300% ROI" could realistically be anywhere from 80% to 520%
- Better methodology exists. It starts with measuring what you can actually observe, not what you hope to achieve
40-60%
of projected ROI typically realised in year one of enterprise AI deployments, driven by methodology flaws, not AI underperformance
Source: McKinsey, The State of AI in 2024
Where the Numbers Go Wrong
Inflated Baselines
The most common trick, usually unintentional, is inflating the baseline cost of the process AI is replacing. If a claims processing team costs $2 million per year and AI reduces processing time by 30%, the projected saving is $600,000. But that 30% reduction in processing time doesn't translate directly to a 30% reduction in cost. The team still exists. The infrastructure still runs. The saving is real but smaller than the headline number.
Vincent has a useful frame for this: the difference between marginal and average cost savings.
Marginal savings are typically 40-60% of the average cost projection. The gap is not a failure of the AI system. It is a failure of the cost model to account for fixed costs, transition periods, and the non-linear relationship between time savings and cost savings.
Dr Vincent Russell
Machine Learning (AI) Engineer
Missing Confidence Intervals
A projected ROI of 300% sounds definitive. But every projection has uncertainty. If the model's accuracy varies by plus or minus 5%, and the volume of processed items varies by plus or minus 15%, and the cost assumptions vary by plus or minus 10%, the actual ROI range is enormous.
Vincent ran a Monte Carlo simulation on a typical enterprise AI business case. The headline projection was 280% ROI. The 95% confidence interval was 90% to 470%. Both the vendor's optimistic projection and the sceptic's pessimistic view were within the confidence interval. The number itself told you almost nothing.
Cherry-Picked Time Horizons
Year-one ROI looks different from three-year ROI. Some AI investments have high upfront costs and compound returns over time. Others show quick wins that plateau. Vendors naturally choose the time horizon that looks best. A project with 50% year-one ROI and 400% three-year ROI will be marketed as the latter. A project with 200% year-one ROI that plateaus will be marketed on the former.
Ask which time horizon the projection uses, and why.
What Better Measurement Looks Like
Start With Observable Metrics
Instead of projecting dollar savings from assumptions, measure what you can actually observe:
- Processing time per unit (before and after)
- Error rate (before and after)
- Throughput (before and after)
- Customer satisfaction scores
- Staff time reallocation (where did the saved time actually go?)
These are empirical measurements, not projections. They're smaller numbers but they're real.
Build the ROI From Measured Data
Once you have three to six months of observed performance data, you can build an ROI model grounded in reality. The measurement period costs patience but produces defensible numbers.
Report Ranges, Not Points
A mature AI ROI report should include:
- Best case, expected case, and worst case scenarios
- Sensitivity analysis showing which assumptions drive the most variation
- Confidence intervals on the key metrics
If the vendor or internal team can only give you a single number, the analysis isn't rigorous enough to base a significant investment on.
The Compound ROI Problem
Enterprise AI ROI compounds over time, which makes single-year projections misleading in both directions. Year one might underperform projections because of implementation friction, training, and workflow adjustment. Year three might significantly outperform because the AI system is now integrated into workflows that didn't exist when the projection was made.
The best ROI framework we've seen accounts for this: modest projections for year one, measured actuals feeding into year-two forecasts, and compound effects tracked explicitly rather than assumed.
AI ROI is real. The problem is not that AI doesn't deliver value. It's that the methodology for projecting and measuring that value is immature. Better methodology means better decisions, more realistic expectations, and ultimately more trust in AI investments that genuinely compound.