Measuring the ROI of AI projects: what to track before and after launch

Most organisations now have at least one AI pilot. Far fewer can say, with numbers, what that pilot is worth. The result is a familiar pattern: an impressive demo, enthusiastic feedback, and then months in limbo because nobody can make the case to scale it — or to stop it.

The fix is not a more sophisticated model. It is deciding, before you build, what you will measure and what result would justify the next step. This guide sets out a practical framework for doing exactly that.

Why AI ROI is hard to measure

AI projects have a few traits that make their value easy to overstate and hard to prove:

• The value is often time — minutes saved per task, spread across many people — which rarely shows up directly in a budget line.
• Quality varies, so "it works" depends on which examples you look at.
• Adoption is uneven. A tool that saves time for those who use it does little if most people don't.
• Costs are spread out: model usage, integration, review time, maintenance and change management.

None of these make measurement impossible. They make it essential to plan.

Step 1: Baseline the work as it is today

You cannot measure improvement without a starting point. Before building anything, measure the current process for the specific task the AI will support:

• Volume — how many times the task happens per week or month.
• Time per task — measured, not estimated, on a sample of real work.
• Quality — error rate, rework rate, customer complaints or escalations.
• Cycle time — how long from request to completion, including waiting.
• Cost — the loaded cost of the people involved, plus any tools or outsourcing.

A week or two of careful observation on a representative sample is usually enough. Record how you measured, so you can repeat it exactly after launch.

Step 2: Count the full cost

A credible ROI includes all of the costs, not just the model bill:

Human review time is the cost teams most often forget — and the one that changes most as quality improves and AI agents are given more autonomy.

Step 3: Choose metrics that match the value

AI creates value in a few distinct ways, whether it is a generative assistant or a machine learning model working behind the scenes. Pick the ones that fit your use case, and define each precisely.

Efficiency

• Time saved per task, measured the same way as the baseline.
• Throughput — tasks completed per person per week.
• Cycle time — how long customers or colleagues wait.

Quality

• Error or rework rate compared with the baseline.
• Consistency — variation in outcomes across people and days.
• Escalation rate — how often the AI hands off to a human, and why.

Revenue and experience

• Conversion — for example, faster quotes leading to more won deals.
• Response time — and its effect on customer satisfaction.
• Capacity — work you can now take on without adding headcount.

Adoption

• Active users as a share of the people who could use it.
• Usage per user over time — curiosity fades; real value persists.
• Acceptance rate — how often AI outputs are used, edited or discarded.

Step 4: Measure honestly

A few practices make results credible to finance teams and sceptical leaders:

• Compare like with like. Measure the same tasks, at similar volumes, with the same method as the baseline.
• Use a control where you can. Roll out to one team first and compare it with a similar team still working the old way.
• Separate novelty from value. Look at weeks four to eight, not the first enthusiastic week.
• Count the time saved only once. If saved minutes aren't redeployed to valuable work, the financial benefit is smaller than the time saved suggests — say so.
• Report ranges, not single numbers, when samples are small.

Step 5: Turn measurements into a decision

Agree the decision rule before the pilot starts. For example:

• Scale if time per task falls meaningfully, quality is equal or better, and most target users are active after six weeks.
• Iterate if value is clear for some users or task types but not others.
• Stop if quality can't reach the required bar, or adoption stays low after you have addressed the obvious barriers.

Writing this down early protects good projects from endless pilots and protects budgets from projects that won't pay back.

A simple ROI model

At its simplest:

Annual value = (time saved per task × volume per year × loaded hourly cost) + (value of quality and revenue improvements)

Annual cost = run costs + human review time + maintenance (+ build cost spread over the expected life of the system)

ROI = (annual value − annual cost) ÷ annual cost

Keep the assumptions visible and conservative. A modest, well-evidenced return wins more support than an impressive number nobody believes.

Reporting results leaders trust

A good AI results report fits on one page:

1. The task and the baseline — what was measured, and how.
2. What changed — time, quality, adoption, with before-and-after figures.
3. The full cost — including review time and maintenance.
4. What we learned — where it works, where it doesn't, and why.
5. The recommendation — scale, iterate or stop, with the next milestone.

The bottom line

AI ROI is not mysterious; it is just usually unmeasured. Baseline the work before you start, count every cost, pick metrics that match the value you expect, measure honestly against a comparison, and agree the decision rule in advance. Do that, and your AI programme moves from promising pilots to decisions you can defend.

Planning an AI initiative and want the business case built in from the start? Let's talk — we design the measurement alongside the product.