When ROI Isn't Money

A recent AI project reminded me that the right measure of return depends on what you are actually trying to prove.

A few weeks ago, I was sitting with the leadership team of a healthcare organization discussing an operational problem that had become increasingly difficult to ignore.

They process a large volume of billing disputes every week. Responding to them requires experienced people to interpret the objection, review clinical and administrative documents, find the right evidence, understand the applicable rules, and prepare a response that can be defended.

What caught my attention was that this was not a broken operation. They had knowledgeable people, established processes, and years of accumulated experience.

Their problem was growth.

As volume increased, maintaining the same level of quality required more human effort. At one point, the person responsible for the process told us that, if the current growth continued, they would probably need to add several people to the team over the next twelve months.

That naturally led us to an AI opportunity.

Could AI interpret the objection, examine the documents already submitted, find the relevant evidence, and prepare a response for an experienced auditor to review?

The idea made sense. And eventually we reached the question every CEO should ask:

What is the ROI?

The problem was that I couldn't answer it honestly.

At least not yet.

The temptation to manufacture an ROI

We had enough information to build a convincing spreadsheet.

We knew the approximate workload. We had an estimate of the additional people they might need. We could estimate employment costs, assume how much work AI could automate, and turn those assumptions into annual savings.

We could have produced an ROI percentage pretty quickly.

But the most important variable was something we had not demonstrated yet: how much of the work could AI actually assist reliably?

Some cases seemed relatively straightforward. The evidence needed to respond was already somewhere in the documents submitted with the bill. Others required an experienced auditor to reconstruct what happened across different clinical encounters or investigate information outside the original documentation.

If we assumed AI could assist 70% of the cases, the economics would look great. At 30%, they would look very different. And if auditors still had to spend almost the same amount of time verifying the AI's work, the apparent automation might create very little value.

The spreadsheet could calculate the answer.

It couldn't tell us whether the assumptions were true.

We were asking the wrong question

That changed the conversation for me.

The initiative did not yet need to prove how much money it would generate over the next three years. It first needed to prove that AI could perform a meaningful part of the workflow reliably enough to change the operation.

So we started thinking about a different question:

What meaningful result must this stage produce to justify continuing?

For this project, the first return is not revenue. It is evidence.

Can the system correctly interpret the objection? Can it locate supporting evidence? Can it show the auditor where that evidence came from? Can it recognize when there isn't enough information instead of inventing an answer? And does it materially reduce the work an experienced person needs to do?

Those are things we can measure.

And if the answer is no, I would rather learn that early than spend another six months defending an attractive financial projection.

Financial ROI is often the end of a chain

This experience reminded me that we sometimes talk about ROI as if it were a single event.

In innovation, it rarely is.

The way I now think about this particular project is:

Learning → Capability → Operational Impact → Economic Impact → Financial Return

First, we learn whether the problem is actually solvable.

Then we demonstrate a capability: the system can perform a meaningful part of the workflow with enough reliability to be useful.

After that, we can measure operational impact. Maybe an auditor who needs twenty minutes for a case can complete it in ten. Maybe the same team can process significantly more cases. Maybe backlog stops growing even while volume continues to increase.

Now we have something we can translate into economics.

The organization might absorb growth without adding people at the same rate. It might respond faster, reduce rework, or recover money sooner.

Eventually, those effects become financial ROI.

Skipping directly to that last number doesn't make the business case stronger. It just hides the assumptions between the idea and the outcome.

Capacity can be a return

This became especially clear when we talked about people.

The organization wasn't looking to reduce its existing team. It was anticipating that increasing volume would require additional people.

So the question wasn't:

“How many jobs can AI replace?”

It was:

“Can we process significantly more work without our human structure growing at the same rate?”

I think that's a much more interesting question for growing companies.

Imagine a team whose workload is growing 30% a year. If maintaining service levels requires the team to grow at roughly the same rate, operating costs follow volume almost linearly.

Now imagine technology allows that same team to handle materially more work while maintaining quality.

Nobody has been replaced. Revenue may not change that month. But the economics of the operation have changed.

The company has created capacity.

For a growing business, the ability to decouple volume growth from headcount growth can be an extremely meaningful return.

We just shouldn't claim its financial value before demonstrating that the capacity actually exists.

This isn't really an AI lesson

The more I thought about it, the more I realized this applies to many decisions founders and executives make.

We implement a CRM and immediately ask how much revenue it generated. We launch a leadership program and want financial ROI. We redesign a process and expect to see the result directly in margins. We build an MVP and expect its first version to validate the economics of the entire business.

Sometimes that makes sense.

Sometimes we're confusing the eventual objective with what the current stage is supposed to prove.

An MVP may need to prove that customers care enough about a problem to change their behavior. An internal system may first need to demonstrate adoption and reduced cycle time. An AI capability may need to prove reliability before anyone makes assumptions about savings or revenue.

None of this means innovation gets a free pass from economic scrutiny.

Actually, I think it means the opposite.

As investment increases, the evidence supporting it should become stronger.

Every stage should earn the next investment

This is the principle I took away from the experience:

Not every stage needs to produce financial ROI. Every stage needs to earn the right to continue.

The first investment may buy learning, but that learning should remove an important uncertainty.

The next may create a capability, but that capability should work reliably in a real workflow.

Then we should see operational impact: more capacity, less time, fewer errors, lower backlog, or better decisions.

Eventually, those improvements need to become economically meaningful.

And as the evidence becomes stronger, we can increase the investment.

If the evidence stops improving, we should be willing to stop.

To me, that's more disciplined than either extreme: demanding a fully quantified financial return before experimenting or continuing to fund experimentation indefinitely because “we are learning.”

AI makes this even more important

AI is particularly good at creating impressive demos.

Give a model a complex document and watch it produce an intelligent answer in seconds. It's easy to extrapolate from there.

If an employee spends thirty minutes doing this and AI does it in thirty seconds, multiply the difference by thousands of cases, multiply again by labor cost, and suddenly you have a spectacular ROI slide.

But a demo is not an operation.

What happens when documents are incomplete? When evidence is contradictory? When information sits in another system? When the model is uncertain but sounds confident? Or when an expert still spends fifteen minutes validating an answer that took the AI thirty seconds to produce?

Those questions are not obstacles to the ROI calculation.

They are what the first stage should help us answer.

That's why, in this project, we're not starting with an automation percentage. We want to measure how many cases can actually be assisted, how much human correction remains necessary, how much time is reduced, and how much additional capacity the team gains.

Then we'll have the inputs for a financial model we can actually believe.

ROI isn't always money — yet

I still believe meaningful business investments eventually need an economic rationale.

What I question is whether we should pretend to know that financial return before we've proven the assumptions underneath it.

Sometimes the first return is learning. Sometimes it's capability. Sometimes it's capacity, time, adoption, fewer errors, or lower risk.

And eventually, it should become money.

The discipline is knowing what needs to become true at each stage for the next investment to make sense.

So the next time someone asks, “What's the ROI?” while an initiative is still surrounded by uncertainty, I would ask one question before opening the spreadsheet:

What meaningful result must this stage produce for us to justify the next investment?

Answer that honestly, and the financial ROI becomes much easier to calculate when the assumptions underneath it become facts.