You're 90 days into your AI pilot. The team says it's going well. The dashboards look positive. But is it actually heading toward production, or is it heading toward a slide deck that says "phase 2 pending" for the next three years?
Here's a five-question diagnostic. Answer honestly. It takes five minutes and might save you six months.
Question 1: Who Owns It After the Pilot Team Leaves?
If the answer is "we haven't discussed that yet," your pilot is already in trouble.
Production AI needs an owner: someone accountable for model performance, data quality, and business outcomes after the pilot team moves on. If that person hasn't been identified by day 90, the pilot was designed to demo, not to ship.
The Ownership Test
Name the person who will be responsible for this system on day 180. If you can't, stop and fix that before continuing.
Question 2: Is It Running on Real Data?
"Real data" means production data, with all its messiness, gaps, and edge cases. Not a curated dataset. Not a cleaned sample. Not synthetic data.
If your pilot is still running on curated data at day 90, you've proven that AI works in ideal conditions. You haven't proven it works in yours.
62%
of AI pilots still use curated datasets at the 90-day mark
Source: Deloitte, Enterprise AI Readiness Survey 2024
Question 3: Can Someone Outside the Pilot Team Explain What It Does?
Walk up to someone in the business unit that's supposed to use this thing. Ask them to explain what the AI does, when they'd use it, and how they'd know if it's wrong.
If they can't answer all three, you have a technology project, not a business capability. The gap between "the pilot team understands it" and "the business understands it" is where pilots go to die.
Question 4: What Happens When It's Wrong?
Every AI system makes mistakes. The question isn't whether yours will. It's whether you've built the processes to catch, correct, and learn from errors.
At day 90, you should have:
- A defined escalation path for incorrect outputs
- A feedback mechanism for users to flag issues
- A monitoring system that detects drift before users do
- A retraining plan for when performance degrades
If you have none of these, your pilot is optimised for the demo, not the real world.
Question 5: Does the Next Project Get Easier?
This is the most important question. If your AI pilot succeeds but teaches the organisation nothing reusable, you've solved one problem and created a precedent for solving every future problem from scratch.
The value of an AI pilot isn't just what it delivers. It's what it teaches the organisation about how to deliver the next one.
Tim Hatherley-Greene
Chief Operating Officer
At day 90, ask:
- Are we building shared infrastructure or single-use infrastructure?
- Is the pilot team documenting patterns that other teams can reuse?
- Will the data pipelines serve more than just this use case?
- Are we building organisational capability or just a solution?
Scoring
5 "yes" answers: You're on track. Keep going.
3-4 "yes" answers: Fixable gaps. Address them now before they become structural.
1-2 "yes" answers: Your pilot is heading toward the graveyard. Pause, reset, and rebuild with production in mind.
0 "yes" answers: You don't have a pilot. You have a demo. That's fine, but stop calling it a pilot and manage expectations accordingly.
The 90-day mark is your last comfortable off-ramp. After this, organisational momentum makes it politically difficult to pause or redirect. Use this window. Be honest about where you are. The best thing you can do for your AI ambitions is kill a pilot that isn't going anywhere, and redirect that investment toward one that is.
- What if my pilot passes some questions but not others?
- Focus on the failures. A pilot with great technology but no ownership plan (Question 1) will stall at handover. A pilot with strong ownership but curated data (Question 2) needs a data reality check. Address the weakest areas first. They're your scaling bottlenecks.
- Is 90 days too early to evaluate?
- No. 90 days is exactly the right time. Earlier and you're still in setup. Later and organisational momentum makes it hard to course-correct. The 90-day checkpoint is about direction, not destination. You're checking the trajectory, not demanding a finished product.