A dashboard that says "320 calls reviewed this quarter" tells a board how busy the compliance team was. It says nothing about what customers experienced. Under the Consumer Duty, that is the question the board has to answer — so the MI has to be built around it.
Activity versus outcomes
Activity metrics — calls reviewed, files checked, training completed — are easy to produce and easy to make look good. Outcome metrics are harder: did customers understand the key risks? Were costs explained? Were vulnerable customers supported? Call QA is one of the few places where outcome evidence can be gathered directly, because the conversation is where the advice was given.
The core call QA measures
| Measure | What it tells you | Watch out for |
|---|---|---|
| Coverage | The share of in-scope calls assessed, and excluded calls by reason. | A high compliance rate on low coverage is weak evidence. |
| Compliance rate | How often rules were met, overall and by call type. | Averages hide the tail. Show the distribution and the failures behind it. |
| Most-missed rules | Where process, training or scripts need to change. | A rule missed everywhere may be a badly written rule. |
| Needs review | Calls that need a person's judgement, and how quickly they get it. | A growing backlog means findings arrive too late to act on. |
| Overrides | How often reviewers changed a result, on which rules and in which direction. | Frequent overrides on one rule usually mean the rule needs rewording. |
| Vulnerability | Signals found by driver, and how consistently they were handled. | Counting flags without the response says little about outcomes. |
| Outcomes by adviser | Where coaching will make the biggest difference. | Small numbers per adviser swing wildly. Use enough calls to be fair. |
Mapping the measures to the four outcomes
Most firms structure Consumer Duty reporting around the four outcomes. Call-derived measures slot into that structure naturally:
- Products & services: needs and objectives captured; alternatives discussed; recommendations linked back to objectives.
- Price & value: fees, charges and long-term costs explained on the calls where they apply.
- Consumer understanding: key risks covered; understanding checked in the customer's words.
- Consumer support: vulnerability identified and handled; clear next steps; follow-ups completed.
Designing MI for a board
- Lead with the conclusion. One sentence per outcome: are customers getting good outcomes, and how do we know?
- Show the trend. A quarter on its own is noise. Twelve months shows whether actions are working.
- Pair every chart with a table. Boards and auditors want the numbers behind the picture — and a table view makes charts accessible to everyone.
- Break it down. By outcome, call type and customer group, including customers with characteristics of vulnerability.
- State definitions once and keep them stable. If a rule changes mid-year, say so, or the trend will mislead.
- End with actions. Each finding needs an owner, a date and, next time, a result.
Common traps
- Sample bias. If only certain calls are reviewed — long ones, or one team's — the MI describes those calls, not the firm.
- Moving goalposts. Changing rules without re-baselining makes improvement impossible to see.
- Traffic lights without thresholds. Red, amber and green mean nothing unless the board has agreed what each colour triggers.
- Precision you don't have. A compliance rate calculated from twenty calls shouldn't be reported to one decimal place.
From dashboard to board report
The annual board assessment should be a summary of a year's MI, not a scramble to assemble it. If the monthly or quarterly dashboard is built around the four outcomes with stable definitions, the board report largely writes itself. We set out a structure in what to put in your Consumer Duty board report.
COSA's analytics show compliance rate, outcomes by call type and by adviser, and the rules most often missed — each with a table view and filters by period, call type and agent. See the MI in the platform.
This article is general information, not legal or regulatory advice.
