How I evaluate an AI project before saying yes

Heads up: this is a draft / placeholder post that AJ will rewrite in his own voice. The structure stays, the words change.

Most failed enterprise AI projects don’t fail because the model was wrong. They fail because the problem was wrong, the data was wrong, or the rollout was wrong. After a few years of being on both sides of that conversation, I have a short list of questions I ask before saying yes.

1. Can a non-AI version of this work?

If the answer is “yes, sort of” — start there. A rules-based baseline, even a clumsy one, tells you whether the problem framing is right. If a simple version helps people, an AI version probably will too. If a simple version doesn’t, an AI version usually won’t either — it’ll just be more confident about being wrong.

2. What does “good” look like, in numbers?

Not “accurate” or “fast.” A specific number, measured the way the business actually measures things. “We reduce the average ticket resolution time from 14 hours to 4 hours” is a goal. “It’s an AI chatbot” is a feature.

3. Who owns the data on day one?

If the answer involves more than two people, or the words “we’re still finalizing access,” the project is going to be late. Not because data access is hard — because data access is political, and political problems compound.

4. What happens when the model is wrong?

Most enterprise AI failures are failure-mode failures. You can ship a 92%-accurate model and still get fired if the 8% is people’s salaries. Decide the recovery path before you decide the model.

5. Who is the executive sponsor — and when did you last talk to them?

If the answer is “we’re going to schedule a kickoff,” the project doesn’t have a sponsor yet. It has a champion. Champions cannot fund.

---

None of this is novel. The reason I write it out anyway is that under the time pressure of a sales cycle, it’s easy to skip — and skipping it is almost always the reason the engagement is hard six months later.