Every AI journey starts somewhere. For most organisations, that starting point is a pilot. A contained experiment to test whether AI can deliver value in a specific context.
The problem is that most pilots are set up to fail. Not because the technology does not work, but because the pilot itself was never designed to prove anything useful.
What bad pilots look like
The science project. Someone in the team builds something clever using ChatGPT or an internal tool. It is impressive in a demo. Leadership nods approvingly. But it was built by one person, for one person, with no connection to how the organisation actually works. When that person moves on, the pilot dies.
The vendor showcase. A technology vendor runs a proof of concept using their platform. It works beautifully in the demo environment. But it was never tested against real data, real workflows, or real users. The organisation learns nothing about its own capability.
The checkbox exercise. Leadership wants to be seen doing AI. A pilot is commissioned, a report is written, a success is declared. But nothing changes. No capability was built. No workflows were improved. The pilot existed to satisfy a board paper, not to create value.
These pilots are not free. They cost time, attention, and something harder to recover: organisational trust in AI. Every failed or meaningless pilot makes the next initiative harder to fund and harder to staff.
What good pilots look like
A good pilot is designed with three things in mind:
1. It solves a real problem
Not a theoretical problem. Not the most exciting problem. A real problem that people experience today, that matters enough for them to invest time in solving, and that is specific enough to measure.
"Improve customer service with AI" is not a pilot scope. "Reduce the time Guest Operations spends answering product-specific enquiries from 20 minutes to under 5 minutes" is a pilot scope.
2. It builds toward something bigger
A good pilot is not a dead end. The architecture, the prompts, the workflows, the team capability built during the pilot should all be usable in what comes next.
If you build a knowledge assistant for one team, the retrieval architecture should be extensible to other teams. If you train people on prompting, the techniques should transfer to different use cases. If you establish governance practices, they should scale.
This is the compounding principle: every step delivers value and unlocks what comes next.
3. It transfers ownership
The team that will use the capability long-term is involved from the start. Not as observers. As participants. They help define the requirements, they test the outputs, they learn the techniques, and by the end, they can extend the capability independently.
A pilot that ends with "the consultants left and we need them back to make changes" has failed, regardless of what the technology does.
The cost of getting it wrong
Bad pilots do not just waste the direct investment. They create three hidden costs:
-
Trust erosion. Teams become sceptical. "We tried AI and it did not work" becomes the narrative, even if the pilot was poorly scoped rather than the technology being inadequate.
-
Opportunity cost. The time spent on a meaningless pilot is time not spent on something that could have delivered real value. In a fast-moving space, six months of wasted effort is significant.
-
Capability stagnation. A bad pilot builds no internal capability. The organisation is no better positioned to do AI work after the pilot than before it. The next initiative starts from zero.
Start with the right questions
Before commissioning a pilot, ask:
- What specific problem are we solving, and for whom?
- How will we measure success?
- What capability will the team have at the end that they do not have today?
- What comes after this pilot if it succeeds?
- Who will own and extend this after the pilot ends?
If you cannot answer these clearly, you are not ready for a pilot. You are ready for a discovery workshop. And that is a perfectly good place to start.