When automated call QA disagrees with an experienced reviewer, the cause is more often the rule than the technology. "Adviser explains the product clearly" means something different to everyone who reads it — including an AI. Rules that are specific and observable get consistent results; vague ones don't.
Five kinds of rule
| Rule type | Checks | Example |
|---|---|---|
| Verbatim | That specific words were said | "This call is recorded for training and compliance purposes." |
| Non-verbatim | That a topic was covered, however it was phrased | Interest roll-up and its effect on the estate explained |
| Vulnerability | Signs of vulnerability, and whether the adviser adapted | Bereavement disclosed; pace slowed and support offered |
| Objection handling | Whether a concern was dealt with in substance | Worry about inheritance acknowledged and answered |
| Document checks | That what was said matches the documents on file | Figures discussed match the illustration |
Verbatim rules suit wording your firm scripts. Everything else is usually better as non-verbatim: customers and advisers rarely use the exact words in a manual, and a rule that demands them will fail good calls.
Principles for rules that work
- Describe observable behaviour. Could a reviewer point to the words in a transcript that pass or fail it? If not, rewrite it.
- One thing per rule. "Explains roll-up, fees and early repayment charges" produces a fail that doesn't tell anyone which part was missing. Split it into three.
- Say what "passed" sounds like. A short description, and where helpful an example, gives both reviewers and the AI a shared standard.
- Scope it to call types. A fact-find rule shouldn't fail an initial enquiry for not covering what was never meant to be covered there.
- Avoid adjectives nobody can measure. "Clearly", "properly" and "adequately" invite disagreement. Say what was explained, not how well.
- Handle legitimate exceptions. If a point doesn't apply in some circumstances — a customer with no dependants, say — write that into the rule.
- Link it to the requirement behind it. A reference to the policy or regulation makes the rule easier to maintain and to explain.
From vague to testable
| Vague | Testable |
|---|---|
| Adviser explains the product clearly | Adviser explains that interest is added to the loan and the balance grows over time |
| Adviser treats vulnerable customers appropriately | Where the customer mentions a health condition, bereavement, financial difficulty or reliance on others, the adviser acknowledges it and adapts — for example, by checking they're comfortable to continue, offering a second call or involving someone they trust |
| Alternatives are considered | Adviser asks about downsizing, savings and other borrowing, and records why each was discounted |
| Customer understands the advice | Adviser asks the customer to explain a key feature in their own words, and corrects any misunderstanding |
| Legal advice covered | Adviser explains that the customer will receive independent legal advice from a solicitor before the plan completes |
Testing rules before you rely on them
- Build a calibration set. Twenty to fifty past calls that your most experienced reviewers have assessed, including some known failures.
- Run the rules and compare. Look at every disagreement and the evidence behind it.
- Fix the rule, not the result. Most disagreements come from wording. Tighten it and run the set again.
- Watch overrides after go-live. A rule reviewers keep overriding in the same direction needs rewording.
Governance
- An owner for every rule, usually in compliance.
- A change history: what changed, when and why.
- Reassessment of past calls when a change matters, so MI stays comparable.
- A regular review as products, scripts and regulation move on.
In COSA, your compliance team writes rules in plain English, chooses how each is judged and which call types it applies to, and can reassess calls when standards change. See how rules work in the platform.
This article is general information, not legal or regulatory advice.
