AI pilot

An AI pilot tests, on one real and narrowly bounded process, whether an AI application holds up in daily operation. What separates it from a prototype is one condition: a pilot runs with real data and real users, and it ends in a decision to scale or to stop.

How do you scope a pilot properly?

Around a process, not a department. Not "AI in customer service" but "assign incoming service requests to the right category and enrich them with customer history". The difference sounds small and decides the outcome, because only the second version is measurable.

Four properties make a good candidate. The process is frequent, otherwise measurement takes too long. It has a checkable result. The damage from a single error is contained. And someone does it today who can judge whether the machine is getting it right.

What makes a bad candidate is just as clear: rare cases, an ambiguous definition of success, high cost per error, and nobody with time to evaluate the output.

What has to be settled before starting

Two things, and both get skipped routinely.

The definition of success, with a number. How many cases should the system handle correctly without escalation, and how is that measured? Without that number a pilot ends in a discussion of opinions.

The baseline. How well and how fast does the process run today? Measure it beforehand or you cannot demonstrate an improvement afterwards, and that demonstration is the basis for every further approval.

Alongside those come the boundary conditions: which data the system may see, where a human decides, and how long the pilot runs. These are the same questions AI governance asks anyway, here applied to one concrete case.

Why pilots fail

The MIT "GenAI Divide" study from 2025 found that around 95% of GenAI pilots show no measurable effect on results. The reasons are rarely technical.

The most common is missing integration. A system that works in a test environment and cannot reach the real systems in daily use was never a pilot. It was a demo.

The second most common is scope: conceived too broadly, therefore measurable nowhere. The third is the missing handover to people, which leaves the process stuck at the first edge case. How to avoid that is covered under human-in-the-loop and in Shipping AI that survives production.

How long a pilot should run

Long enough for a dependable number, short enough for a decision. In practice that is six to ten weeks, and the duration follows the case volume rather than the calendar.

The useful calculation is on volume. If a process occurs fifty times a week, four weeks give you two hundred evaluated cases, and that will carry a decision. At five cases a week the same statement takes ten months, which tells you it is the wrong candidate.

Two dates belong in the plan from the start. A checkpoint at roughly the halfway mark, where adjustments are allowed. And an end date, on which a decision is made.

The end date is the more important one. A pilot with no end becomes a permanent condition where a system is half used, nobody trusts it, and the decision never comes. And if you cannot produce a number when the time is up, that is a result too: the process was not measurable, which is a finding about the scope.