Adjusting for the Economy: Overlays on Credit Models | HL Hunt
Adjusting for the Economy: Overlays on Credit Models
A credit model knows the period it was fitted on. When conditions change, it keeps predicting as though they hadn't — and rebuilding and validating a replacement takes months. Overlays fill the gap: a scalar on predicted default, a moved cutoff, a tighter threshold for one segment. They're necessary, and they're also where the least governed decisions in underwriting tend to accumulate. Overlays go on quickly when things worsen and come off slowly when they improve, so without an expiry date, a temporary adjustment quietly becomes the model.
What you'll learn
Why models need them
Per our validation analysis, a model is fitted on a development sample that ended some time ago, and it predicts the relationships that held then.
| What changes | How fast | How fast a model can follow |
|---|---|---|
| Economic conditions | Weeks | Months to rebuild and validate |
| Support programmes starting or ending | On a known date | Only after outcomes appear |
| Your own channel mix | A quarter | After new vintages mature |
| Data definitions, per our attribute analysis | Overnight | After someone notices |
The right column is the problem. Outcomes take months to observe, per our loss forecasting analysis, and a model can only learn from outcomes it has seen. Per our correlation analysis, a model fitted on a benign period has never observed a severe one and can't have learned what happens in it.
An overlay is how human judgment enters in the gap — which makes it both essential and the part of the decision process most in need of discipline.
The evidence that justifies one
Two legitimate triggers, and one that isn't.
- Divergence. Per our monitoring analysis, recent vintages performing materially worse — or better — than the model predicted, on early indicators, across more than one cohort.
- A known change the data can't contain. A support programme ending on a set date, a sharp move in conditions, a change in a key data source — something you can see coming and the model structurally can't.
- Not: general unease. An overlay applied because conditions "feel" riskier, without a calculation, is a policy change dressed as a model adjustment — and should be governed through credit policy, with its own rationale, rather than slipped into the scoring layer.
Divergence needs one more check before it justifies anything. Per our channel analysis, worse performance across the book can come from a change in who you're lending to rather than from the model decaying. If within-channel performance is stable, the fix is in the mix, not in an overlay.
The double-counting check
The most common technical error, and it tightens twice for one cause.
Many model inputs already respond to changing conditions. When households come under pressure, utilization rises, recent delinquencies appear, and inquiries increase — and a model using those attributes produces worse scores without any adjustment.
An overlay on top of that counts the same deterioration twice: once through the inputs, once through the scalar. The approved population shrinks by more than the risk change justifies.
What to check before applying one:
- Has the score distribution of applicants already shifted? If so, the model is already responding.
- Is the divergence in calibration or in ranking? A model that still ranks correctly but predicts levels too low needs a level adjustment, not a reordering.
- Is actual performance worse than the current scores predict, not worse than last year's scores predicted?
Question three is the precise test. Compare outcomes to what the model predicted for those same accounts at origination. If the model's own predictions are still right, the deterioration is in the applicants rather than in the model, and the model is handling it.
Choosing the form
| Form | What it does | Use when |
|---|---|---|
| Probability scalar | Multiplies predicted default | Calibration off, ranking intact |
| Cutoff shift | Raises or lowers the approval line | Risk appetite has changed |
| Segment-specific adjustment | Applies to one segment only | The divergence is concentrated |
| Line or amount cap | Limits exposure, not approval | Loss severity is the concern |
| Additional rule | A new knock-out | Rarely — hardest to remove |
Choose the narrowest form that addresses the evidence. A divergence in one segment doesn't justify a portfolio-wide scalar, and a concern about loss severity is better addressed with amounts, per our line analysis, than by declining people.
New knock-out rules deserve particular suspicion as overlays. Per our record retention analysis, blunt rules tend to be the least nuanced part of a decision process — and a rule added in a crisis is rarely anyone's job to remove afterwards.
Sizing it
A worked example of a probability scalar, stylized:
| Figure | |
|---|---|
| Model-predicted default rate, recent vintages | 4.0% |
| Observed rate on early indicators, same vintages | 5.2% |
| Ratio | 1.3 |
| Approval cutoff, predicted default | 6.0% |
| Applicants newly declined | Model PD between 4.6% and 6.0% |
A 1.3 scalar with an unchanged 6% cutoff declines everyone whose unadjusted prediction was between 4.6% and 6%, since 4.6% × 1.3 ≈ 6%. That band can be a large share of approvals — which is why an overlay's size should be derived and stated rather than chosen because it feels proportionate.
Three disciplines for sizing:
- Derive it from the observed divergence, as above, and write the calculation down.
- Use early indicators with care, since per our cure analysis early delinquency partly cures — an overlay sized on early arrears alone overshoots.
- Record it in the decision record, per our audit trail analysis, so every decision it touched can be reproduced with and without it.
Who it removes
The fairness question, which an overlay can raise even when the underlying model has been tested.
An overlay changes the approved population, and it may not change it evenly. A scalar that pushes a band of applicants below the cutoff removes whoever sits in that band — and if that band is disproportionately composed of particular groups, the overlay has a disparate effect the model alone didn't.
What to do:
- Test the overlay's effect on approval rates across groups, per our fair lending analysis, before it goes live.
- Consider less discriminatory alternatives — a line cap rather than a decline, a segment adjustment rather than a portfolio one.
- Document the business justification alongside the calculation.
- Watch overrides, per our override analysis: reviewers overriding the overlay frequently are telling you something about its calibration.
Getting it off again
The asymmetry that makes overlays a governance problem rather than just a modelling one.
Tightening feels prudent and loosening feels risky. An analyst who proposes an overlay in a downturn is being careful; one who proposes removing it is taking a chance with their name on it. So overlays accumulate on the way down and linger on the way up.
What makes removal happen:
- An expiry date set when the overlay is applied, forcing a review even if nobody asks for one.
- A removal test defined in advance — for example, recent vintages performing within a stated tolerance of the unadjusted model for two consecutive quarters.
- A named owner responsible for the review.
- A register of every active overlay, with its rationale, size, date, and expiry.
- A test cell, per our testing analysis: a small share of applications decided without the overlay shows directly whether it's still earning its place.
Item five is the evidence that makes removal defensible. Without it, the only information about declined applicants is that they were declined — per our measurement analysis, you never see how they'd have performed, so the overlay can never be shown to be unnecessary.
The overlay nobody remembers
An adjustment applied during a past downturn, with no expiry and no owner, is still declining applicants years later for reasons nobody can state. If your decision process contains an adjustment whose rationale isn't written down, you've found one.
When everyone overlays at once
The market-level consequence, per our monoculture analysis.
Lenders reading the same data respond to the same deterioration at the same time. Each overlay is individually sensible; together they contract credit across the market simultaneously — and per our shock analysis, that contraction arrives precisely when households need access most, which can deepen the conditions the overlays respond to.
A single lender can't fix that. It can:
- Size overlays from its own evidence rather than from what competitors are doing.
- Prefer amount limits to declines, which preserve access while controlling exposure.
- Remove overlays promptly when its evidence says so — which, in a market still tightened, is also where the best risk-adjusted lending is.
The third point is the commercial one. A lender whose overlays come off on evidence while competitors' linger is lending into a market where good applicants are being declined elsewhere for stale reasons.
Overlays with an audit trail and an expiry date
HL Hunt AI Underwriting applies overlays as versioned, documented adjustments with owners and expiry dates, records every decision with and without them, runs test cells to measure whether each is still needed, and reports their effect on approval rates across groups.
Frequently asked questions
An adjustment applied on top of a model's output — a scalar, a cutoff move, a segment threshold — bridging the gap until a rebuilt model reflects new conditions.
When predictions diverge from outcomes, or when a known change is coming that the training data can't contain. General unease is a policy question, not an overlay.
No documentation, no removal, double-counting what the inputs already capture, and uneven effects on who gets approved — plus synchronized tightening across lenders.
According to a test set when it was applied, with an expiry date that forces a review and a small test cell that shows whether it's still needed.
Key takeaways
- Overlays bridge the months between a change in conditions and a model that reflects it — and they need more discipline than the model does.
- Check within-channel performance first; worse results can be mix shift rather than model decay.
- Compare outcomes to what the model predicted for those same accounts; if it's still right, an overlay counts the deterioration twice.
- A 1.3 scalar at an unchanged cutoff removes a whole band of applicants — derive the size and write it down.
- Test an overlay's effect on approval rates across groups before it goes live.
- Set an expiry, a removal test, an owner, and a test cell at the moment of applying it, because tightening outlives the reason for it.
Know every adjustment in your decision process
Get started with HL Hunt AI Underwriting for an overlay register, reproducible decisions with and without each adjustment, and test cells that show when an overlay can come off.
This guide is educational and does not constitute legal, compliance, or model risk advice. Worked figures are stylized illustrations. Model risk management expectations, fair lending requirements, and documentation obligations for adjustments to credit decisions vary by institution type and jurisdiction. Consult your model risk, compliance, and legal functions before applying or removing any overlay.