AI Call Auditing: Score Every Sales Call Against Your Playbook

AI call auditing means an AI scores every sales call transcript against a scorecard you wrote, so a manager reviews the exceptions, not a random few. Use five to eight yes-or-no questions, test them on calls a manager already scored, and get recording consent first.
This builds on AI lead scoring, which decides who gets the call, and on AI lead generation, which builds the list. To build the whole system with your team, see my AI sales enablement page.
What is AI call auditing?
AI call auditing is the automatic review of recorded calls. A transcript goes in, a scorecard comes out, and every rating comes with the evidence from the call. The scorecard is the same one a manager would use, applied to every call instead of the handful they have time to hear.
You will see two names for it. “Call auditing” is the term contact centers use, and most of the pages that rank for it are about customer-service quality and compliance. “Call scoring” is the sales version. Same method, different questions. This post is about the sales version: did the rep follow your playbook, and what should the rep change on the next call.
Why score every call instead of a sample?
A sample tells you about the calls you picked. Every call tells you about the team.
In Claap’s own call data from October 2025 to October 2026, 27% of the teams that talked about reviewing calls said managers review only a sample. Claap notes that these teams had already booked a call with it, so this is a vendor’s sample of its own buyers, not a survey of the market. Treat it as a hint.
The reason it matters is simple. If a manager hears four calls a week, the other forty are invisible. A rep can skip the agenda, never ask who signs, and end calls with “let’s catch up sometime” for a month before anyone notices.
I cold called for over two years, and it taught me to have no fear of picking up the phone. That is also why I take the call itself so seriously. You can have the best product in the world, but if you have the worst salesperson, you won’t win the business. Scoring every call is how you find out which one you have.
What goes on a sales call scorecard?
Start from your playbook, not from a template. A scorecard is your playbook turned into questions. If the playbook says every discovery call should end with a dated next step, the scorecard asks whether it did.
Here is a starting scorecard for a discovery call. I adapted it from the discovery-call examples in Claap’s guide. Each question is about one behavior, and the answer is yes or no with a quote from the call.
| Question | Why it is on the list |
|---|---|
| Did the rep set an agenda and confirm how much time they had? | Calls without an agenda drift |
| Did the rep find out what the problem costs the buyer? | A problem with no cost rarely gets a budget |
| Did the rep learn who decides and who signs? | Deals stall when the signer never joined |
| Did the rep ask about timing and what is driving it? | Tells you whether to chase or wait |
| Did the call end with a dated next step? | “Let’s catch up sometime” is not a next step |
| Did the rep stay under 50% of the talking time? | My own example: a balance check, not a rule from a study |
Three rules for writing the questions. Gong’s help center and Revenue.io’s scorecard guide agree on most of them.
- One behavior per question. Gong tells admins to split any question with several parts into separate questions.
- Name who does the action, and prefer facts to opinions. “Did the rep ask about timing?” can be checked in the transcript. “Was the rep persuasive?” cannot.
- Keep the scale small and define it. Gong suggests ranges such as 0 to 2, with each value spelled out, and warns that its AI tends to answer yes. A yes-or-no question with a rule for what counts as yes is harder to inflate.
Keep it to one call type at a time. Claap’s guide recommends 5 to 8 criteria per call type. A cold call, a discovery call and a demo each get their own scorecard.
How do you set up AI call auditing in six steps?
You can run this with a spreadsheet and an AI assistant before you buy anything. The counts below (20 calls, three score bands) are my suggestions, not a standard.
- Pick one call type and one goal. For example: discovery calls, and getting reps to end every one with a dated next step.
- Write the playbook down, then turn it into questions. At a B2B software company I helped grow, we did not train people one-on-one. New hires got a written onboarding doc and training videos, and they were productive within days. The same idea applies here. If the playbook lives in a manager’s head, the AI has nothing to score against.
- Have a manager score 20 calls by hand. Revenue.io suggests two reviewers score the same sample independently and compare answer by answer. It calls this a starting exercise, not a statistically valid sample size, and I agree.
- Run the AI on the same 20 calls. Compare its answers with the manager’s. Where they differ, the question is usually the problem, not the AI. Rewrite the question, add an example of what counts as yes, and run it again.
- Decide what happens with a score. AI scores every call. The manager reads the lowest and the strangest, and gives one rep feedback on one change, within a day or two while the call is fresh (my suggestion). Claap’s guide describes the same hybrid, with a weekly review habit.
- Re-check monthly. Whenever we disagreed, we looked at the data, and whatever the data said won, not who yelled the loudest or who had the fanciest title. Compare scores with real outcomes, and fix the questions that do not predict anything.
You can test the idea today with a prompt like this in Claude or ChatGPT:
Here is my scorecard and a transcript of one sales call. Answer each question yes or no. For each answer, quote the line from the transcript that proves it. If the transcript does not show it, say “not shown”. Do not guess.
The quotes matter. A rep will argue with a score. A rep will accept a score that shows the sentence it came from.
Once the prompt works, save it so it does not drift. In my Claude skills video I show how a reused chat can forget its original instructions by the tenth run, and how a skill, which is just a folder with one text file, keeps the template the same every time. A scorecard is the same kind of thing.
Video · 15 min
If You're Not Using Skills, You're Using Claude Wrong
I explain how to turn instructions you keep re-typing into a reusable skill, which is how I would keep a call scorecard the same from call 1 to call 100.
Watch the video →
Do you need permission to record sales calls?
Yes, in many places. Get consent before you record, and before any tool joins the call as a notetaker. This is general information, not legal advice. Ask a lawyer about the states your reps and buyers are in.
Two rules show how different the answers are:
| Rule | What it says | Source |
|---|---|---|
| Federal law | A private person who is a party to a call may record it, or may record it when one party has given prior consent. The exception does not apply if the recording is made to commit a crime or a tort | 18 U.S.C. 2511(2)(d), Cornell LII, read November 17, 2026 |
| Florida | Interception is lawful when all of the parties to the communication have given prior consent. Violating the statute is a felony of the third degree | Fla. Stat. 934.03(2)(d) and (4)(a), read November 17, 2026 |
So a rep in a one-party state can often record, but a call with someone in Florida needs everyone’s consent. For a call between two states, I would follow the stricter rule and ask a lawyer. That is my inference, not a legal finding.
In practice, do four things:
- Say it at the start. “This call is recorded so we can train our team. Is that okay?” Then wait for the yes.
- Put it in writing. Add it to the meeting invite, the booking page and your terms.
- Honor a no. If someone says no, stop recording and take notes.
- Check your tool’s settings. Know where transcripts are stored, who can see them and how long they are kept, before you paste a real call into any AI service.
I tell people to get an AI notetaker like Granola for their phone and computer and record every meeting, demo, pitch and call, so you can feed everything back into Claude. The condition is that you have the consent first.
Should you use a call tool or your own scorecard?
Pick by how many calls you have and how much setup you want.
| Option | How it works | Choose it when |
|---|---|---|
| Manager listens and scores in a spreadsheet | A person scores a sample | You have under a few dozen calls a month |
| Notetaker plus your own AI prompt | You paste each transcript and your scorecard into an AI assistant | You want to test your scorecard before paying for software |
| Conversation-intelligence tool (Gong, Avoma, Claap and others) | The tool records, transcribes and scores each call automatically | You have steady call volume and reps who work from one system |
Gong’s help center shows what the automatic version looks like. Its AI Call Reviewer can suggest answers to a human reviewer, or score every matching call on its own with the label “Scored by Gong AI”. It cannot score voicemails or calls with fewer than 100 words in the transcript, and repeat reviews of the same call may produce slightly different answers. Those limits are normal for this kind of tool. Ask any vendor for them in writing.
What can go wrong with AI call auditing?
- The AI says yes too often. Gong says so about its own tool. Write questions with a clear rule for yes, and test them on calls you know were bad.
- The scores shift between runs. Gong says repeat reviews may differ slightly. Treat a score as a flag for a person to check, not a verdict.
- The playbook is wrong. Revenue.io puts it plainly: a rubric is a coaching aid, not a validated predictor of win rate. A perfect score on a bad playbook is still a bad call.
- Reps feel watched. Tell the team why you are doing it before the first score goes out. Show them their own calls first.
- A score with no coaching. One simple piece of advice can beat a long report. I once coached a struggling salesperson with one simple piece of advice: put your phone on Do Not Disturb and hide all alerts and notifications. He became the number one salesperson in the entire company and stayed number one for years. Pick the one change that matters and leave the rest.
How do you know AI call auditing is working?
Track four numbers each month, starting from a baseline you measured before you turned it on.
| Metric | What it tells you |
|---|---|
| Average score by rep, per question | Which behavior to coach next |
| Share of calls ending with a dated next step | Whether the playbook is being followed |
| Agreement between AI and manager on the weekly sample | Whether you can trust the AI score |
| Win rate for calls scoring high and low | Whether the scorecard predicts anything |
If you cannot show a link between high scores and wins after a few months, change the questions. For how to run this as a small trial, see how to run an AI pilot project, and for how to judge whether the time is paying off, see how to measure AI ROI.
What should you do next?
This week, do three things. Pick one call type. Write five to eight yes-or-no questions from your playbook. Get a manager to score 20 recorded calls by hand, with consent, and compare them with an AI assistant’s answers. You will know within a couple of days whether the questions are clear.
For the message that follows a call, see AI sales emails. If you are still choosing where else AI fits in your sales work, the guide to AI use cases lists the first tasks worth testing, and AI automation covers how to turn a manual routine into a repeatable one.
If you want a person to write the scorecard, set up the scoring and train the team to coach from it, start on my AI sales enablement page or my AI consulting page.
Sources
- Claap, Call scoring: how to score sales calls, published October 14, 2026 and updated October 28, 2026, read November 17, 2026. Source of the 5 to 8 criteria per call type, the hybrid review model, the 27% figure from Claap’s own call data and the discovery-call examples.
- Gong Help Center, Answer scorecard questions with AI Call Reviewer, read November 17, 2026. Source of the manual and automatic modes, the 100-word and voicemail limits and the note that repeat reviews can differ.
- Gong Help Center, Configure scorecard questions, read November 17, 2026. Source of the advice on one behavior per question, small scales and the AI’s tendency to answer yes.
- Revenue.io, Build a scorecard, published February 20, 2026, read November 17, 2026. Source of the two-reviewer comparison and the line that a rubric is a coaching aid, not a validated predictor of win rate.
- Cornell Legal Information Institute, 18 U.S.C. 2511, read November 17, 2026. Source of the party and one-party consent exception.
- Florida Senate, Florida Statutes 934.03, 2025 edition, read November 17, 2026. Source of the all-party consent rule and the third-degree felony penalty.


