What changed in the business after the AI budget was spent?

The board paper looks good. AI usage is up. More employees have access. There’s a long, colourful list of pilots. Every number in it may be accurate.

And the one question that matters is still unanswered: what changed in the business because of the money we spent?

It helps to pull three things apart. Adoption tells you whether people use the capability. Task performance tells you whether the work got better. Realised benefit tells you what the organisation did with that improvement. Getting from one to the next takes evidence and a decision, and most board papers stop at the first.

Usage is not value

Usage is worth tracking. It shows interest and helps diagnose friction. But frequent use doesn’t prove quality, and broad access doesn’t prove value. An assistant that drafts documents might save time, or create extra review work, or simply produce more material nobody needed.

For each significant initiative, name the task and what “acceptable” looks like. Compare the old process with the AI-supported one on representative work, including corrections, rejected outputs and cases handed back to a person. Measure the whole task, not just the step that was easiest to count.

The National Bureau of Economic Research’s Generative AI at Work studied a customer-support setting and found the effects differed between workers. Both the setting and the variation matter. One workflow’s result shouldn’t become a productivity assumption for an unrelated function.

Follow saved time to an actual benefit

If a task now takes less time, ask what happened to the freed capacity. Maybe the team served more customers, cleared a backlog, improved quality or changed its staffing plan. Each of those is a different operational choice with a different kind of evidence.

Don’t book estimated hours saved as cash unless a budget actually changed. Equally, don’t dismiss a real service improvement just because nobody’s headcount went down. Match the claim to what happened: capacity, quality, speed, lower risk or a financial saving.

Put a business owner against every benefit and agree when it will be reviewed. Technology teams can measure how the service behaves, but the operating team controls most of the choices that turn improvement into value. A benefit without an owner is hard to tell apart from a hope.

Show the running costs and the limits

Include recurring model and infrastructure costs, source maintenance, support, evaluation and review effort. Where several use cases share a foundation, explain how its cost is allocated rather than charging the full amount to each one.

Report constraints next to outcomes. Permission gaps, missing data owners or too little review capacity can block expansion even when a model performs well. The board needs the constraint that affects the next investment, not a catalogue of every technical issue.

The Office for National Statistics has researched how management practices relate to technology and AI adoption. The evidence is observational, so it shows association, not cause. The practical question still stands: does your organisation have the management capacity to turn a tool into a dependable process?

End every review with a decision

Put each initiative into plain categories: expand a proven workflow, fix a specific weakness, continue a bounded experiment, or stop. Don’t let pilots stay “promising” forever.

When you expand, say which evidence supports it and which assumptions change at scale. A new geography, user group or data source can bring a completely different operating problem. The plan should include who evaluates those changes and what it costs.