We came close to buying the tool that cost ten times more. It would have been the easier approval, too, and that’s the part I keep thinking about.

TL;DR

We almost bought the expensive one

This summer I put a countdown on the click. My argument in The End of the Click was that AI answers are replacing the search results page, and that the questions still worth winning are the ones that come right after the clinical one: where do I get treated for this, and who’s the best doctor for that? Those questions don’t name us. Someone asks an AI for a cardiologist near them, and either our physicians, services, and locations come up in the answer or they don’t.

That isn’t a niche behavior anymore. A KFF poll this spring found that about one in three U.S. adults had used an AI chatbot for health information in the past year.

So the first thing we needed was a way to measure it. My team did what a good team does. We ran a long, careful evaluation of the established enterprise SEO platforms and how well each one measures visibility in AI search. They had every feature you could ask for. They also cost roughly ten times what we ended up spending. We had a recommendation, and we were close to moving forward with the leader.

Then a small startup reached out, unprompted, and we took the call.

I’d love to tell you we had a framework that pointed us to it. We didn’t. The startup’s pitch didn’t do it either. The question changed. Talking with a company focused narrowly on providers made us look at our own situation again, and the answer was obvious once we asked it: our provider presence is where we have the most room to improve, and it’s what patients care about most.

The big platforms answered “what can AI search measurement do?” We needed the answer to “do our doctors show up when patients ask?” The second question was the right one. It also cost a tenth as much to answer.

What I put in front of the committee

Any AI tool we bring in goes through a cross-functional AI steering committee, a group of leaders from across the organization who review it for risk. That’s where this pilot went next.

I didn’t lead with the return. I led with no PII, no PHI. No personally identifiable information, and no protected health information, the patient data HIPAA covers. The tool would only see what’s already public about our providers and locations, and it wouldn’t connect to any of our systems.

Then I fenced it. One of the conditions we agreed to: if we ever want to connect the tool to anything of ours, through an API or anything else, that’s a separate approval. The yes covered exactly what we asked for, and nothing it might grow into.

I did explain the return. What I didn’t have were specific, measurable goals, because that review usually happens before there’s a contract to attach them to. So I committed to defining them in the pilot’s statement of work, and that was enough.

The real sticking point was the vendor’s size. It’s a small startup. In a large health system, “small startup” sounds a lot like “might not be here next year.” What settled it was how the vendor got classified: the lowest-risk tier, a vendor we’d like to have rather than one supplying something the organization can’t run without. If the startup disappeared tomorrow, we’d lose a measurement tool. Patient care wouldn’t notice.

A yes from governance isn’t a business case

Later I asked myself a question I didn’t enjoy answering: would the ten-times platform have been a harder yes?

No. Probably easier. It’s an established industry leader, and it’s SEO, so there’s obviously no patient data anywhere near it. The contract would have been bigger, but the approval would likely have been the same or faster. The cheaper option that fit better was the harder one to get through governance, because the thing governance worries about is risk, and a startup carries more of it.

That isn’t a flaw in the committee. It told us up front that its role was AI governance, full stop, and it didn’t ask to set the pilot’s goals. That’s the right line to draw. Governance that starts judging business value tends to do both jobs worse: it slows down the safe ideas and lends its credibility to the expensive ones.

A yes from governance tells you it’s safe. It doesn’t tell you it’s worth it.

Here’s the catch. Once the risk review passes, the only person in the process checking whether the thing is worth the money is the business owner. If that’s you, and you take the committee’s yes as proof you have a good case, nobody else is going to catch it.

Run your own ask through both checks before you take it anywhere.

The two checks

Who's actually checking your AI ask?

Describe something you want to bring in. Governance judges it left to right. You judge it bottom to top. They aren't the same check.

Data it touches
Connects to your systems
The vendor
Size of the ask
Proof you have

Easy yes, no case

Safety check
Likely a quick yes
Value check
Needs results you already measured. You have a demo.

This is where the ten-times platform sat. Governance will probably say yes, and the yes will feel like a business case. It isn't one. Nobody in that room is checking whether this is worth what it costs, so you'll have to.

Make the ask small enough to approve →

Shrink it

The map moves with JavaScript on. What you're seeing is the ten-times platform from the story above.

Runs in your browser. Nothing is saved or sent.

Make the ask small enough to approve

Here’s how I’d sum up what we did: don’t build the full business case before the pilot. Build the pilot so it produces the business case.

The pilot is small enough that, as the business leader who owns the budget, I can make the call. That’s part of why I didn’t need a polished return-on-investment model to get started. A bigger agreement is a different conversation. I own that budget too, but I won’t commit to a larger engagement without at least an implicit nod from my own leadership, and I’ll want to earn it with results rather than projections.

That’s the order I’m trying to get right. The proof should grow with the size of the ask. When the bigger ask comes, I expect to walk in with a baseline, goals we agreed on, and data showing whether we hit them. Asking for the big version first, with nothing but a vendor’s demo behind it, is how teams end up defending a contract they can’t measure.

I didn’t walk in with a business case. I walked in with a way to get one.

Shared goals, on purpose

The first thing the pilot does is set a baseline for every measure we care about. Only then do we agree on improvement targets, and we’ll be negotiating those with the vendor, not handing them over.

I want the vendor set up to succeed, and a vendor can’t be held to a goal it doesn’t control. Plenty of the work that improves our visibility won’t be theirs. Some of it is ours. Some of it isn’t even Marketing’s: it belongs to the providers themselves. A tool can show us where a physician is missing from the answers. It can’t do the parts only that physician can do, and our physicians are busy taking care of patients.

So the goals are shared by three parties: the vendor, my team, and our providers. If you write a pilot agreement as though the vendor alone can deliver the result, you’ve set it up to fail and given everyone an easy person to blame when it does.

What if it doesn’t work?

This is the question most writing about AI business cases skips, because it assumes the pilot works. Ours might not, at least not the way I hope.

If it falls short, I’m not starting with the tool. I’m starting with us. Did we move the needle on the tasks we control? If not, why not? Then the providers: did they understand their part? Were they bought in and taking part, or passive, waiting for Marketing to handle it?

I’ll ask those at checkpoints along the way, not just at the end. If we’re doing our share and the providers aren’t, I don’t cancel. I change course and make sure they understand why their participation decides the result.

And here’s what makes this a safe bet for a cost-conscious organization: even falling short is worth something. At the end of the pilot we’ll know exactly how visible our providers are in AI search today, and knowing where you stand already puts you ahead of a lot of the field. The risk is fenced and the cost is small. Even the worst case pays for a baseline we didn’t have. That’s an easy yes, and it’s honest, because nobody has to oversell anything to get it.

I’ve learned what happens when an AI project gets approved on a promise. This time I asked for less and put the fence in writing. The bigger ask can come later, with the pilot’s numbers behind it.

If you’ve taken an AI pilot through governance recently, I’d like to know what your committee got stuck on, and who in your process is actually checking whether it’s worth the money. Head over to LinkedIn and let’s talk about it.