Call QA sample size calculator
How many calls should you monitor?
Your numbers
Your result
Reviewing 50 of 3,000 calls a month, you have a 64% chance of catching at least one call with a failure that affects 2% of calls.
For a 95% chance, review 145 calls — 4.8% of the calls you handle a month.
- Chance of catching it
- 64%
- with 50 calls reviewed
- Calls to review
- 145
- for a 95% chance
- Share of your calls
- 4.8%
- of 3,000 a month
Affected calls nobody hears
On average, 59 of the 60 calls a month with this failure fall outside your sample.
- Calls with the failure
- 60a month
- In your sample
- 1a month
- Never reviewed
- 59a month
Assumes calls are picked at random and the failure is spread evenly across the period. Each kind of failure — each disclosure, each vulnerability driver — faces these odds on its own.
Assess every call, then sample the passes
COSA assesses every in-scope call against your own rules, and people review what's flagged. A small random sample of the calls that passed then checks the automated review isn't missing anything.
Method
How the calculation works
A sample can answer two questions — will it catch a problem at all, and how precisely does it measure one — and they need very different sample sizes.
- Chance of catching at least one
- 1 − (1 − failure rate) ^ calls reviewed
- Calls needed
- ln(1 − confidence) ÷ ln(1 − failure rate), rounded up
- From a known number of calls
- 1 − chance that every call drawn is clean, drawing without replacement
- Affected calls unheard
- failure rate × (calls handled − calls reviewed)
Detection asks whether, if something is going wrong, you will find at least one example of it. Estimation asks what proportion of calls meet your standards, and whether that is getting better. This calculator answers the detection question — the one about stopping harm. Estimation needs much bigger samples.
If a share of calls contain a failure, the chance that any one call chosen at random is clean is one minus that share. The chance that every call in your sample is clean is that figure multiplied by itself once for each call reviewed, and the chance of catching at least one example is one minus the result. The sample you need is the smallest one that pushes the chance of missing the failure below one minus your chosen confidence.
When you enter how many calls you handle, the calculator works out the exact chance of drawing from that fixed pool without replacement: each call reviewed is one fewer that could hide the failure. It counts the failing calls as the rate times your volume, rounded to whole calls. The answer is slightly lower than the standard figure, and the gap only grows large when you review a big share of all calls. If the rate times your volume rounds to zero, the period isn't expected to contain a failing call at all.
Worked examples
What the numbers look like
Calls chosen at random from a very large pool, as in our full explanation of call QA sampling.
| Calls with the failure | 25 calls | 100 calls | For 95% | For 99% |
|---|---|---|---|---|
| 0.5% (1 in 200) | 12% | 39% | 598 | 919 |
| 1% (1 in 100) | 22% | 63% | 299 | 459 |
| 2% (1 in 50) | 40% | 87% | 149 | 228 |
| 5% (1 in 20) | 72% | 99% | 59 | 90 |
- One call in fifty, 25 reviewed. 0.98 multiplied by itself 25 times is about 0.60, so a failure that affects 2% of calls goes completely unseen about six times in ten. Even 100 calls miss it more than one time in eight.
- A 95% chance of catching it. ln(0.05) ÷ ln(0.98) ≈ 148.3, which rounds up to 149 calls — about 150. For a failure that affects 1% of calls, it is 299: about 300.
- A known call volume. At 3,000 calls a month, a 2% failure means 60 affected calls. Drawing from that fixed pool, 145 calls give a 95% chance — a few fewer than 149.
- Estimating a rate. With 50 calls reviewed, the 95% margin of error on a 10% failure rate is about ±8.3 points: the true rate could be anywhere from about 2% to 18%. Knowing you handle 3,000 calls a month barely changes that (±8.2 points).
Limits
What a sample can't tell you
Which advisers need support
Boards and team leads usually want to know which advisers need support, and sampling makes that very hard. Spread 50 calls across ten advisers and each is judged on five calls. An adviser who gets one call in ten wrong will show no problems at all in their five calls nearly 60% of the time — and a single bad call can make a good adviser look like an outlier.
A sample big enough to say anything about individual advisers is usually bigger than the whole QA budget.
What this calculator assumes
- It assumes calls are chosen at random from all calls. Real samples often aren't — particular teams, call lengths or weeks get reviewed more — which makes them less representative than the figures suggest.
- It covers one kind of failure at a time, across the whole firm. Every separate issue you want to detect — each disclosure, each vulnerability driver — faces the same odds.
- It tells you whether a sample is likely to contain an example, not how widespread the problem is or who caused it.
- Margins of error use the normal approximation, which is rough for small samples and for rates close to 0% or 100%.
- It is general information, not legal or regulatory advice.
With COSA
Every call first, then a sample
The FCA doesn't prescribe a coverage figure. The Consumer Duty does require firms to monitor the outcomes customers receive and evidence them, and a firm relying on a sample should be able to explain why the sample is big enough to do that.
COSA assesses every in-scope call against your own rules, with the quote and reasoning for every verdict. People review what's flagged. A small random sample of passes gets a human review too, to check nothing is slipping through and to keep reviewers calibrated, and every excluded call is logged with its reason. The sample stops being your evidence and becomes your quality check on the evidence.
It is how Compare Retirement moved from sampling 26% of calls to 100% oversight. See how the platform works, or model the reviewer hours with the call QA ROI calculator.
FAQ
Questions about call sampling
How many calls should I sample to catch a 1% failure?
About 300. To have a 95% chance of catching at least one example of a failure that affects 1% of calls, you need to review 299 randomly chosen calls; for a 99% chance, 459. For a failure that affects 2% of calls, the figures are 149 and 228. Estimating a rate precisely, or saying anything about individual advisers, needs more.
Does the FCA set a minimum number of calls to monitor?
No rule sets a coverage figure. The Consumer Duty requires firms to monitor outcomes and evidence them, and a firm should be able to explain why its monitoring is adequate. When every call can be assessed automatically, a small manual sample becomes harder to justify.
Why does a small call sample miss rare failures?
A rare failure may simply not be in the calls you chose. If a failure affects one call in fifty, a random sample of 25 calls has a 60% chance of containing no example at all, and even a sample of 100 misses it entirely more than one time in eight.
Does the number of calls we handle change the sample size?
Only a little, unless you review a large share of them. The standard figures assume a very large pool of calls. Drawing from a known number without replacement needs slightly fewer: with 3,000 calls a month, 145 calls give a 95% chance of catching a failure that affects 2% of calls, against 149 from the standard formula.
How accurate is a compliance rate taken from a small sample?
Not very. With 50 calls reviewed, a reported 10% failure rate could really be anywhere from about 2% to 18%, so a quarter-on-quarter improvement from 12% to 9% on samples that size is indistinguishable from noise.
Make your sample a check, not your evidence
In 30 minutes we'll show COSA assessing calls like yours, and what your compliance team would review each morning.
- Tailored to your call types
- No obligation
- Reply within 24 hours