Let’s talk about something the AI industry doesn’t like discussing: ROI measurement. Research from McKinsey and Gartner consistently shows that most AI ROI figures you hear are somewhere between optimistic and fictional.

I’ve seen project proposals claiming 500% ROI based on assumptions that wouldn’t survive ten minutes of scrutiny. I’ve also seen genuinely valuable AI projects that struggle to demonstrate their worth. The measurement problem is real.

Here’s how to think about AI ROI more rigorously.

Why AI ROI Is Hard to Measure

Several factors make AI ROI genuinely challenging:

Attribution complexity. If revenue increases after an AI deployment, was it the AI? Market conditions? Other initiatives? Sales team effort? Isolating AI’s contribution is rarely straightforward.

Productivity gains are fuzzy. If someone saves 30 minutes per day, does that translate to value? Only if they do something valuable with that time. And measuring that is difficult.

Comparison baselines shift. AI projects often run alongside process improvements. Comparing “with AI” to “without AI” requires a control group that usually doesn’t exist.

Costs are hidden. The AI tool license is the visible cost. The integration, training, ongoing support, and productivity loss during adoption are often untracked.

The Honest Framework

Despite these challenges, you can still measure AI value. It just requires intellectual honesty.

Step 1: Define Specific, Measurable Outcomes

“Improve productivity” isn’t measurable. “Reduce time to complete customer quote from 4 hours to 30 minutes” is measurable.

Before any AI project, define success in terms that can be verified. Good outcome definitions are:

• Quantified (numbers, not adjectives)
• Observable (you can actually see whether it happened)
• Attributable (you can reasonably connect the outcome to the AI)
• Timebound (when you’ll measure)

Bad outcome: “AI will make our team more efficient.” Good outcome: “Quote generation time will reduce from 4 hours to 1 hour within 3 months of deployment, measured by average completion time in our CRM.”

Step 2: Establish Baseline Before Deployment

You can’t measure improvement without knowing where you started. Before deploying AI:

• Measure current performance on your defined outcomes
• Track the measurement for long enough to understand normal variation
• Document the measurement methodology
• Store the baseline data where it won’t be lost or modified

Many organisations skip this step in their eagerness to deploy. Then they can’t demonstrate value because they have no comparison point.

Step 3: Track Costs Completely

Include all costs, not just license fees:

Direct costs:

• Software licenses
• Infrastructure (compute, storage)
• Implementation services

Indirect costs:

• Internal staff time on implementation
• Training time
• Productivity loss during transition
• Ongoing support and maintenance
• Management overhead

Hidden costs:

• Data preparation work
• Integration maintenance
• Model drift remediation
• Compliance and governance

Many AI ROI calculations only include direct costs. The real picture is usually 2-3x higher.

Step 4: Measure Post-Deployment Honestly

After deployment, measure the same outcomes using the same methodology. Watch for:

Hawthorne effect: People perform better when they know they’re being measured. Initial improvements may not sustain.

Cherry-picking: Resist the temptation to count successes and ignore failures. Measure everything.

Moving goalposts: Don’t change the outcome definition to make results look better.

Regression to mean: Early outlier results (good or bad) often moderate over time.

Step 5: Calculate ROI with Appropriate Humility

Once you have real data:

Simple ROI: (Benefits - Costs) / Costs

But present ranges, not single numbers. Given measurement uncertainty, “ROI between 80% and 150%” is more honest than “ROI of 115%.”

Acknowledge what you can’t measure. Some AI benefits (employee satisfaction, brand perception, risk reduction) are real but hard to quantify.

Common ROI Pitfalls

Counting gross savings, not net. “We saved 1,000 hours” isn’t value unless those hours converted to something valuable. Were people redeployed to higher-value work? Was headcount reduced? Did you avoid hiring?

Ignoring opportunity cost. The money spent on AI could have been spent elsewhere. Was this the best use of those resources?

Double-counting benefits. If AI reduces quote time AND improves win rates, don’t count both if the win rate improvement is because of faster quotes.

Extrapolating pilots. Pilots often perform better than full rollouts because they get extra attention and involve the most capable users. Scale-up results are typically worse.

What “Good” AI ROI Looks Like

Based on projects I’ve seen with honest measurement:

Productivity tools (Copilot, etc.): 10-30% productivity improvement for tasks where the tool is applicable. Not 10-30% improvement in total productivity – improvement on specific tasks.

Document processing: 50-70% reduction in processing time for high-volume, repetitive document tasks. Accuracy comparable to or better than humans.

Customer service AI: 20-40% reduction in handling time, 10-20% improvement in first-contact resolution. Customer satisfaction impact varies widely.

Predictive analytics: Highly variable. 5-15% improvement in forecast accuracy is common. Business value depends entirely on how forecasts are used.

These numbers are meaningful but not transformative. Beware vendors or consultants promising dramatically higher figures.

My Recommendation

For any AI investment over $100k:

1. Define measurable outcomes before starting
2. Establish baselines with real data
3. Track all costs including hidden ones
4. Measure honestly post-deployment
5. Report ranges with acknowledged uncertainty

This won’t give you impressive slides for the executive committee. It will give you actual understanding of whether your AI investments are working.

In a world of AI hype, honest measurement is a competitive advantage. It helps you double down on what works and cut what doesn’t. That’s worth more than a compelling but fictional ROI number.