Alarm Fatigue: Why More Sensitive Monitoring Catches Less | HL Hunt
Alarm Fatigue: Why More Sensitive Monitoring Catches Less
Every monitoring system faces the same trade: catch more, or send fewer alerts that turn out to be nothing. Moving a fraud system from 80% to 95% sensitivity catches 45 more cases and produces 2,492 more false alarms, which is 55 to 1, and precision falls from 32.5% to 8.7%. At 8.7% precision, dismissing an alert is the correct response 91% of the time, so the system teaches its users the behaviour that defeats it.
What you'll learn
The core thesis
Every monitoring system in a financial institution faces the same choice: catch more of the thing you are looking for, or send fewer alerts that turn out to be nothing. Fraud detection, compliance flags, model drift alarms, overdraft warnings, transaction monitoring. The same trade, in each case.
Our position is that the usable value of an alert is its precision rather than its coverage, that systems are almost universally tuned toward coverage because coverage is what gets asked about, and that the resulting false positive volume destroys the response capability the system was built to create.
The mechanism is not subtle. People who dismiss nineteen alerts in a row dismiss the twentieth, and the twentieth is the real one.
What tuning for coverage actually does
Take 100,000 transactions with a fraud base rate of 0.3%, so 300 genuine cases.
| Tuning | Caught | False alarms | Alerts to review | Precision |
|---|---|---|---|---|
| 80% sensitivity, 99.5% specificity | 240 | 499 | 739 | 32.5% |
| 95% sensitivity, 97% specificity | 285 | 2,991 | 3,276 | 8.7% |
The second configuration catches 45 more cases. It also produces 2,492 more false alarms.
That ratio is the figure nobody computes before loosening a threshold. The question is always framed as how much more we would catch, and per our reference point analysis the answer sounds good on its own, because the cost appears in a different place and in a different unit.
And the arithmetic is worse for rarer events. At a 0.05% base rate with 90% sensitivity and 99% specificity, the system produces 45 true positives and 1,000 false ones, a precision of 4.3%. Per our sampling analysis, this is the same base rate problem that governs sampling: when the thing you are looking for is rare, almost everything you find is something else.
The cost that does not appear anywhere
- Review labour. The only cost that gets counted, and the smallest of these.
- Dismissal becomes the default. At 8.7% precision, dismissing is correct 91% of the time, so the person reviewing learns the correct behaviour and applies it to the exception too.
- Investigation quality falls. A queue of 3,276 gets less attention per item than a queue of 739, so the true positives that are found are worked worse.
- Customer friction. Per our declines analysis, a false positive in a fraud system is frequently a declined transaction for someone who did nothing wrong.
- The alert loses standing. Once a flag is known internally to be mostly noise, it stops escalating, which is the failure the system was designed to prevent.
- Real signals get re-tuned away. The usual response to an unworkable queue is raising the threshold under pressure, which happens without analysis and without a record.
Item two is the one that makes this different from an ordinary efficiency problem. A system with poor precision does not merely cost more to run. It actively trains its users toward the wrong response, and per our institutional memory analysis that trained behaviour outlives any subsequent retuning because nobody tells the reviewers the precision changed.
A falsifiable claim
Dismissal rates on true positives should rise with queue volume, which means the same genuine case is more likely to be missed when the system is tuned more sensitively. Any institution with alert-level outcomes can test it: identify confirmed true positives and measure what share were initially dismissed by a reviewer, split by the queue volume that week. If dismissal of true positives is flat across volumes, reviewers are unaffected by load and the fatigue mechanism is weak. We expect it to rise, which would mean sensitivity and effective detection diverge past a point, and that a more sensitive system can catch less.
When coverage is the right objective
There are cases where a low precision system is correct, and they are not rare.
- When the cost of a miss is catastrophic and asymmetric. Sanctions screening and certain safeguarding flags are tuned for coverage deliberately, because the consequence of one miss exceeds the cost of thousands of reviews.
- When review is cheap and automatic. A check resolved by a second data lookup rather than by a person has a different economics entirely.
- When the requirement is to look rather than to find. Some monitoring exists to demonstrate that monitoring occurred, per our ambiguity analysis, and a precision argument misses what it is for.
- When the alert is a hold rather than a decision. A flag that pauses something briefly costs less than one that declines it.
- During a new pattern. Per our stacking analysis, a system looking for something it has not seen before should be loose until it has examples.
The first bullet is the genuine counterexample and it should be stated clearly. Where the asymmetry is extreme, precision is the wrong objective and the right question is whether you have the capacity to review everything the system produces. The error is not tuning for coverage; it is tuning for coverage in a system whose consequence is symmetric, which is most of them.
Designing for precision
- State the precision you are targeting before tuning. Not the sensitivity, which is the number everyone quotes.
- Compute false alarms per extra catch for every threshold change. In the example above that is 55, which reframes the decision.
- Size the queue to the review capacity you actually have, because a queue larger than capacity is not a more sensitive system, it is an unreviewed one.
- Tier by consequence rather than by score. An alert that can be auto-resolved should never reach a person, and per our review economics analysis the only cases worth human time are the ones where the decision could change.
- Measure dismissal rate on confirmed true positives, which is the fatigue measure and almost nobody reports it.
- Separate the alerts that must be reviewed from the ones that should be. Mixing a legal obligation into a discretionary queue means the obligation inherits the queue's dismissal rate.
- Retune with a record, per our decommissioning analysis, because threshold changes made under queue pressure are the least documented decisions in any monitoring programme.
Item six is the one with the most exposure attached. Per our prioritization analysis, an obligation expressed as one item in a long discretionary queue is an obligation being handled at the queue's average attention, which is not what anyone intended when they added it.
Item three is the discipline that prevents most of the damage. Tuning sensitivity upward without adding review capacity does not increase detection; it increases the share of alerts that nobody looks at properly, and the detection rate can fall while every reported metric improves.
Scenarios and what we are watching
| Scenario | What we would expect |
|---|---|
| Automated triage improves | Precision at the human queue rises sharply, which is the genuine resolution of this trade |
| Regulatory expectations emphasize coverage | Systems tune toward sensitivity, queues grow, and effective detection falls while reported coverage rises |
| Alert volumes keep rising | Thresholds get raised informally under pressure, with no record of when or why |
| Nothing changes | Institutions continue reporting sensitivity, which is the number that can be improved without improving anything |
The second row is the dynamic worth watching most carefully, and it is uncomfortable. A supervisory question about what a system failed to catch is answerable and a question about what it caught but nobody reviewed is not, so the incentive points toward coverage. Per our measurement analysis, an institution optimizing the measure it will be asked about is behaving rationally, and the result is a system that is more sensitive and less effective, with every number pointing the right way.
Frequently asked questions
In the worked case, 55 extra false alarms per extra case caught, with precision falling from 32.5% to 8.7% and the queue growing four and a half times.
Because at 8.7% precision dismissal is correct 91% of the time, so reviewers learn the wrong response and apply it to the real case too.
Yes, where a miss is catastrophic and asymmetric, where review is automatic, or where the alert is a brief hold rather than a decision.
Precision, false alarms per extra catch, and the dismissal rate on confirmed true positives, which is the fatigue measure nobody reports.
Key takeaways
- The usable value of an alert is its precision, not its coverage, and institutions tune for coverage because that is what gets asked about.
- Moving from 80% to 95% sensitivity bought 45 extra catches at 2,492 extra false alarms, a ratio of 55 to 1.
- Precision fell from 32.5% to 8.7%, at which point dismissing an alert is correct 91% of the time.
- At a 0.05% base rate, a 90%/99% system yields 4.3% precision: when the target is rare, almost everything found is something else.
- Poor precision trains the wrong response, and the trained behaviour outlives any later retuning.
- Coverage is the right objective where a miss is catastrophic and asymmetric, or where review is automatic rather than human.
- Size the queue to real review capacity, because sensitivity without capacity raises the share of alerts nobody examines properly.
- Never mix a legal obligation into a discretionary queue, or the obligation inherits the queue’s dismissal rate.
This analysis is educational and does not constitute compliance, risk, or model risk advice. All figures are stylized illustrations using assumed base rates, sensitivities, and specificities, and do not describe any particular system or institution. Calculations assume independent observations and a constant base rate, which is a simplification. Supervisory expectations for transaction monitoring, sanctions screening, and other detection systems vary by jurisdiction and institution type and may mandate coverage levels irrespective of precision. Consult your compliance and risk functions before changing any monitoring threshold.