AI Pilot Project: How to Run One

An AI pilot project is a short, small trial of AI on one real task, with a written goal and a stop rule set before it starts. Run it for 30 to 90 days, measure it against a baseline, then decide to scale it, fix it or stop it.
This guide walks through each step, with a one-page brief you can copy. For help running one with your team, see the AI consulting page or my fractional Chief AI Officer service.
What is an AI pilot project?
An AI pilot project is a time-boxed test of AI on one task, with the people who do that task today, using their real data. It answers one question: is this worth doing at full scale? A pilot ends with a decision, not a launch party.
Pilots get mixed up with two neighbors. This table keeps them apart. The lengths are my suggested ranges, not an industry standard.
| Stage | The question it answers | Who uses it | Usual length |
|---|---|---|---|
| Proof of concept | Can the AI do this task at all? | A few people, often test data | Days to a few weeks |
| Pilot | Does it work for real people on real work, and is it worth the cost? | A small group, real data, set limits | 30 to 90 days |
| Production | Can we run it every day for everyone? | Everyone it is meant for | Ongoing |
An “AI pilot program” usually means several pilots run under shared rules. Get one pilot right first. The rules you write for it become the template for the rest, and they feed into your wider AI strategy and 90-day roadmap.
Why do AI pilots stall?
Most stalled pilots never had a clear definition of success, so nobody could say whether they worked. Two large sources point the same way.
| Finding | Source |
|---|---|
| At least 30% of generative AI projects would be abandoned after proof of concept by the end of 2025, because of poor data quality, inadequate risk controls, escalating costs or unclear business value. This is a forecast, not a count | Gartner press release, July 29, 2024 |
| About 5% of AI pilot programs reached rapid revenue acceleration, while the vast majority stalled with little to no measurable profit-and-loss impact. The report blamed a “learning gap” in tools and organizations more than model quality | MIT NANDA, The GenAI Divide: State of AI in Business 2025, as reported by Fortune, August 18, 2025 |
Treat both as signals, not exact rates. The MIT figure comes from a preliminary study: interviews with 150 leaders, a survey of 350 employees and a review of 300 public deployments. Gartner’s number is a prediction.
My reading is that the causes on both lists are the kind you can fix before launch: unclear value, no baseline, weak data and no one in charge. That is an inference, not a finding from either source. The rest of this guide is built around fixing those four things.
How do you run an AI pilot project?
Run it in six steps, in this order. The weeks are a suggested schedule for a 60-day pilot, so stretch or shrink them to fit.
| Step | What you do | When |
|---|---|---|
| 1. Pick one task | Choose a repeated task with a clear start and finish | Week 1 |
| 2. Write the one-page brief | Goal, limits, numbers, stop rule, owner | Week 1 |
| 3. Measure the baseline | Time and count the task as it is done today | Weeks 1 to 2 |
| 4. Run the pilot | Real people use the AI on real work | Weeks 2 to 8 |
| 5. Check weekly | Same numbers, same way, every week | Every week |
| 6. Decide | Scale, fix or stop, in writing, on the end date | End date |
How do you pick the right task?
Pick a task that is repeated, has a clear start and finish, can be measured today, and does little harm if the AI gets it wrong. Red Hat’s guide for first pilots gives the same advice: start with a real problem, not with an AI tool, and keep the task small and specific (Deb Richardson, Red Hat Blog, December 16, 2024).
| Good first pilot | Poor first pilot |
|---|---|
| Drafting replies to common customer questions | “Improve customer service” |
| Summarizing sales calls into a standard note | A task nobody does the same way twice |
| Preparing a weekly report from data you already have | Anything where one wrong answer is costly or public |
| A task someone on the team wants help with | A task nobody owns |
Be honest about whether AI is the answer at all. I tell people that not every problem is an AI project. Sometimes the fix is an inventory system, and even a great system is pointless if people will not use it.
Then think small. I aim for really small, easy, quick wins. When a team sees how fast something gets built, they say wow, it was that easy, and they want to automate everything else. A modest first pilot that works does more for your next one than a big one that does not.
For a list of ideas by department, see AI use cases for business. If you are not sure your team and data are ready, run an AI readiness assessment first.
What goes in a one-page pilot brief?
The brief is one page that fixes the goal, the limits and the stop rule before any results exist. One practitioner guide says the same thing in a sentence: criteria set after the results come in do not count (Sagnik Bhattacharya, September 29, 2026). It also suggests that the person who judges the pilot and the person who runs it both sign and date the page.
Copy this layout:
| Line | What to write |
|---|---|
| Task | The one task, in one sentence |
| Problem | Why it matters, in one sentence |
| Group | Who uses the AI, and who does not |
| Tool | The approved tool or tools, and the plan or account type |
| Data | What the AI may read, and what it may never see |
| Baseline | Minutes per task, volume per week and error rate today |
| Success | The numbers that mean “scale it” |
| Stop rule | The numbers or the date that mean “stop” |
| Dates | Start, weekly check-ins and the end date |
| Owner and judge | One person who runs the pilot, one who decides |
Here is an example with made-up numbers, for illustration only. A 12-person services company pilots AI first drafts of replies to common customer questions.
| Line | Example |
|---|---|
| Baseline | 10 minutes per reply, about 40 replies a week, 1 in 10 needs a correction after sending |
| Success | Reply time, including checking, is 6 minutes or less, and corrections do not rise |
| Stop rule | If checking time stays above 8 minutes after week 4, or corrections rise, stop |
| Dates | 60 days, with a check every Friday |
Who needs to be involved?
You need four roles, and in a small company one person can hold two of them. The names are mine, not a standard.
- Owner. Runs the pilot day to day, collects the numbers and reports every week.
- Judge. Decides at the end. This should be someone who can say stop, and ideally not the person who proposed the pilot.
- Users. The people who do the task today. Their feedback decides whether it fits real work, and they also deserve a say in whether it should continue.
- Data and risk check. Someone who confirms what data the AI may see before the pilot starts. In a small company this may be you.
Tell the users what the pilot is for. People who fear that AI is there to replace them will not give honest feedback. The AI adoption guide covers how to roll AI out to a team without that fear.
Free resource · Guide
Mastering Claude
I wrote this guide on setting up Claude and getting real work out of it, so everyone in your pilot group can start from the same place.
Get it free →
How do you set success and stop rules?
Write them before the pilot starts, as numbers, measured the same way every week. A rule you write after the results arrive is just a story about the results.
Use four outcomes, so the decision is never “it was fine, I guess”:
| Outcome | When to choose it | What happens next |
|---|---|---|
| Scale | The success numbers were met and quality held | Plan a wider rollout |
| Fix and extend | Close, with a clear cause such as checking time or a data gap | Change one thing and run again for a set time |
| Redesign | It works but for a different task than you planned | Write a new brief |
| Stop | The stop rule was met | Close it, write down what you learned |
Stopping is an acceptable result. I have been there: I almost started a company selling jewelry software, because my dad was in the jewelry business. We interviewed a bunch of his friends and they all said they already had software they were happy with. After a few months of research I scrapped the idea. Finding that out cost a few months. Finding it out after building would have cost a lot more.
A pilot works the same way. Spending a few weeks to learn that a task is a poor fit for AI is the pilot doing its job.
What should the AI be allowed to touch?
Give it the least access it needs, and make anything risky ask for approval. I always give Claude permission to read, because that does no harm, but I never give it blanket permission to edit or delete. When it wants to write, edit or delete, it asks and I approve. There are horror stories of an AI that deleted someone’s files.
Put these limits in the brief:
- Read-only first. Let the AI read and draft. Let a person send, post, delete or pay.
- Named data only. List the folders, systems or documents it may read. Keep customer details, payroll and anything legally sensitive out of the first pilot unless your rules allow them.
- A person checks the output. Count that checking time in your numbers, because it is part of the real cost.
- A way to switch it off. Know who stops the pilot and how.
For the wider rules a company needs, see the AI governance guide and the AI risks guide.
How do you run it week to week?
Use the same numbers every week and expect the first version to be rough. I have a rule of thumb for this: your first version is going to be bad. It will break, it will not connect, it will not find the email. Do not get frustrated and do not give up, because it gets really good the more you update it. I once spent twelve hours on a Gmail skill and had to scrap the whole thing, but I learned a lot from it.
A simple weekly rhythm:
- Same measures, same way. Record the baseline numbers from the brief. Do not add new ones halfway.
- Watch it at first. When you teach AI a process, watch it the first few times. If it goes the wrong way, say stop, fix it and keep going. After a few runs you can tell whether it still needs you.
- Change one thing at a time. If you change the prompt, the data and the tool in one week, you will not know which helped.
- Log problems and wins. A shared note is enough. It becomes the evidence for the final decision.
- Do not move the goalposts. If the numbers fall short, write down why and keep the rule as it was.
Expect the first weeks to look worse than the baseline while people learn the tool. That is an inference, not a measured result, so judge on the later weeks and check that the trend is improving.
What does a pilot cost?
Count the whole cost, not only the subscription. The same cost groups apply as in any AI project:
| Cost | Examples |
|---|---|
| Tools | Subscriptions, seats, usage fees |
| Setup | Connecting files, writing prompts and templates |
| Training | Time to learn, plus any coach or trainer |
| Checking and fixing | People reviewing and correcting the output |
| Ongoing care | Updating prompts, handling tool changes |
Checking time is the cost most often left out. Add it to the “with AI” number in your baseline comparison. For the full worksheet, see AI ROI: how to tell whether AI is paying off.
How do you decide at the end?
Hold a short meeting on the end date, with the owner, the judge and a few users. Compare the final numbers with the success and stop rules in the brief, and write down one of the four outcomes.
If the outcome is “scale”, do not jump to everyone at once. Write a short plan: who is next, what training they need, who owns it and how you will keep measuring. The AI implementation guide covers that step, and the AI adoption guide covers how to keep people using it.
Whatever the outcome, save the one-page brief and the final numbers. The next pilot starts from them. Some companies keep a short list of tasks waiting for a pilot, and run the next one when the last finishes.
What mistakes sink an AI pilot?
Six mistakes show up again and again:
- No baseline. Without a measured “before”, any result is an opinion.
- A vague goal. “See what AI can do” cannot succeed or fail.
- Too big. A pilot that spans five teams is a rollout in disguise.
- No owner or no judge. Nobody can say stop, so nothing ends.
- Leaving out checking time. The AI looks faster than it is.
- Moving the goalposts. The rule changes when the numbers disappoint.
Who can help run an AI pilot?
You can run a small pilot yourself with the brief above. A senior outside person is useful when you want to pick the task, set the numbers and review the result without bias. Some companies bring in a fractional Chief AI Officer for this. For that role, see what a Chief AI Officer does.
What should you do next?
This week, pick one task and write the one-page brief. Time the task five times without AI and fill in the baseline. Set an end date within 90 days and name the owner and the judge. If you want help choosing the task, setting the numbers and training the team, my fractional Chief AI Officer service covers it, and my AI workshops train the people who will use the AI.
Sources
- Gartner, Gartner Predicts 30% of Generative AI Projects Will Be Abandoned After Proof of Concept By End of 2025, published July 29, 2024, read November 16, 2026.
- Fortune, MIT report: 95% of generative AI pilots at companies are failing, published August 18, 2025, read November 16, 2026. It reports the MIT NANDA study The GenAI Divide: State of AI in Business 2025.
- Red Hat Blog, Deb Richardson, AI use cases: Preparing for your first AI pilot project, published December 16, 2024, read November 16, 2026.
- Sagnik Bhattacharya, AI pilot success criteria, published September 29, 2026, read November 16, 2026. A practitioner guide, not a study.


