Segmentation: When One Model Should Be Several

A single model applied to a mixed portfolio is an average of populations that may behave differently, and where they do, it underperforms on all of them simultaneously. That's the case for segmentation and it's real. What's usually missing from the discussion is the test that determines whether it applies: a segment is justified when the relationships differ, not when the populations do. A group that is simply riskier isn't a segment — it's a variable, and a single model handles it by including one. Most proposed segments turn out to be variables in disguise, and each unnecessary one carries an ongoing cost that falls on a function other than the one proposing it.

The test that decides it

Two groups differ in one of two ways, and only one justifies separate models.

	Different level	Different relationship
What it means	Group B is riskier overall	A predictor works differently for B
Example	Channel B defaults at 9% versus 6%	Utilization predicts strongly for A and weakly for B
Handled by	A variable	A separate model
Cost	One column	A full additional model

Most segmentation proposals describe the left column. "These customers behave differently" almost always means they default at a different rate, which a single model absorbs by including an indicator — and does so more reliably, because it estimates the difference from the full dataset rather than from a fragment.

The right column is different and does require separate treatment, because no single set of coefficients can represent a predictor that matters for one group and not another. A model forced to average two incompatible relationships is wrong for both, in opposite directions.

How to test rather than assume: fit the combined model with interaction terms between the candidate segmentation variable and the main predictors. If the interactions carry meaningful weight and improve out-of-sample performance, the relationships genuinely differ. If they don't, you have a variable.

This takes an afternoon and it precedes any decision to build. Teams that skip it build segments on intuition about populations, which is the wrong question asked confidently.

Where relationships genuinely differ

The cases where interaction testing usually confirms a real difference:

• Thin file versus established file. The strongest case. Bureau-derived variables predict well where there's history and poorly where there isn't — and per our missing data analysis, the imputation layer is doing most of the work for thin-file applicants. Different predictors matter, which is a genuine segmentation.
• Product type. Instalment and revolving behave differently enough that variables predicting one may be uninformative for the other.
• New versus existing customer. For existing customers, behavioural data dominates application data — the finding in our renewal analysis. That's a different predictor set, not a level shift.
• Consumer versus commercial. Entirely different data and different relationships.
• Secured versus unsecured. The exercise cost in our option analysis differs enough that borrower characteristics carry different weight.

And the cases that usually turn out to be variables:

• Geography, unless something structural differs.
• Loan size bands, where the relationship is usually monotonic.
• Channel, which per our cold start analysis shifts the level substantially and rarely changes the relationships.
• Score bands, which is segmenting on the model's own output.
• Time periods, which is a monitoring question rather than a segmentation one.

The pattern: segmentation is justified when the available information differs, not when the population does. Thin-file, new-versus-existing, and consumer-versus-commercial all involve genuinely different inputs. Geography and channel involve the same inputs on different people.

The data constraint

The constraint that most often decides the question regardless of the analysis.

Splitting a portfolio splits the defaults, and defaults are what a model learns from — the count that matters per our vendor analysis.

Worked: a portfolio of 90,000 accounts with a 7% loss rate has roughly 6,300 matured defaults. Split it:

Segment	Share	Defaults	Viable?
Main	72%	~4,536	Yes
Thin file	18%	~1,134	Probably
New product	7%	~441	Marginal
Commercial	3%	~189	No

The bottom two rows are the problem. A segment with a few hundred defaults produces a model that fits noise confidently — and the smaller segments are frequently the ones where segmentation was most wanted, because they're the ones that behave least like the main population.

Which is a genuine bind: the segments that most need their own model are the ones least able to support one. The resolutions:

• Keep them in the combined model with an indicator and interactions, which borrows strength from the whole dataset.
• Use an expert scorecard for the small segment, documented per our cold start analysis.
• Wait until defaults accumulate, which for low-loss segments can be years.
• Buy a model built on a wider population of that type.

The first is usually right and is the least chosen, because it's less satisfying than a dedicated model.

What each model costs

The cost that's routinely omitted, because it falls on functions other than the one proposing the split.

Each model requires its own:

• Development and documentation.
• Validation, initially and periodically.
• Monitoring — the input and performance tracking in our monitoring guide, per model.
• Fair lending analysis.
• Reason code mapping, per our explainability analysis.
• Governance review and approval.
• Implementation and testing.

These costs are largely fixed per model rather than proportional to the volume it covers — the same structure as our institutional cost analysis, operating inside a risk function. A portfolio split five ways carries roughly five times the ongoing burden of one.

Which produces a specific failure: a model covering 3% of volume costs the same to maintain as one covering 60%. If the segment doesn't justify that, the honest answer is to fold it back — and folding models back is something almost no organization does, which is why model counts ratchet upward.

The practical recommendation: price the ongoing cost before approving a segmentation, and set a review that can reverse it. A segment that stops earning its keep should be merged, and nobody will propose that unless the review exists.

Comparing honestly

The methodological trap that makes segmentation look better than it is.

Segmentation always improves in-sample fit. More parameters fit the training data better, necessarily. So any comparison on the development sample will favour segmenting, regardless of whether it helps.

What a valid comparison requires:

1. Out-of-sample evaluation — a holdout the segmented models never saw.
2. Out-of-time evaluation where possible, since segmentation gains are frequently period-specific.
3. Comparison on the same metric, computed the same way.
4. Evaluation on each segment separately, since a gain concentrated in one segment and a loss in another may net to nothing.
5. Comparison against the right alternative — not the naive combined model, but the combined model with an indicator and interaction terms, which is what segmentation must actually beat.

Point five is where most comparisons go wrong. Segmented models are usually compared against a combined model that doesn't include the segmentation variable at all, which is not the relevant alternative — and the gain attributed to segmentation is frequently the gain from including the variable.

Where segmentation does win out of sample, the gain is real. Where it wins only in sample, you've bought several models' worth of ongoing cost for a fitting artifact.

Boundary problems

Segmentation creates edges, and edges have their own failures.

• Discontinuities. Two nearly identical applicants on either side of a boundary can receive materially different decisions, which is hard to explain and hard to defend.
• Gaming. Where the boundary depends on something an applicant or a channel influences, it will be influenced.
• Drift. The mix across segments changes over time, so a segmentation appropriate at build may not be later — and this needs monitoring per segment, not just per model.
• Sparse edges. Applicants near a boundary are thinly represented in both segments' training data, so performance is worst exactly at the edge.
• Assignment errors. An applicant routed to the wrong segment is scored by a model not built for them, and this failure is silent.

The mitigations: check for discontinuities by scoring boundary cases under both models and inspecting the gap, monitor segment mix over time, and prefer boundaries that are structural rather than continuous — thin-file versus established is a genuine categorical difference, while a score cutoff is an arbitrary line through a continuum.

That distinction is worth generalizing: segment on categories that exist in the world, not on thresholds you invented. Categorical boundaries don't produce discontinuity problems because the underlying difference is genuinely discontinuous.

The fair lending question

A segmentation boundary is a decision about who gets assessed by which mechanism, which makes it subject to the analysis in our governance framework.

What to check:

• Whether the segmentation variable correlates with protected characteristics. Thin-file status does, since it correlates with age and with being newer to the credit system.
• Whether outcomes differ by segment in ways that produce disparity.
• Whether each segment's model is individually tested, not just the portfolio in aggregate — a disparity confined to one segment can be invisible in a combined analysis.
• Whether a less discriminatory alternative exists, including the unsegmented option.
• Whether reason codes remain accurate per segment, since a segment-specific model produces segment-specific reasons.

The point worth emphasizing: segmenting on thin-file status is analytically well justified and requires careful handling, because the segment correlates with protected characteristics and the whole purpose is to treat it differently. That's not a reason to avoid it — treating thin-file applicants with a model built for them is generally better for them than scoring them with one that isn't. It's a reason to test outcomes per segment and document the justification, which is a modest requirement attached to a genuinely good decision.

A decision procedure

1. State the hypothesis — which relationships you believe differ, and why.
2. Test for interaction in the combined model before building anything.
3. Count matured defaults in the smallest proposed segment.
4. If interactions matter and defaults suffice, build and compare out of sample against the combined-with-interactions alternative.
5. If interactions matter and defaults don't suffice, keep it combined with interactions or use a documented expert overlay.
6. If interactions don't matter, add the variable and stop.
7. Price the ongoing cost of each additional model.
8. Check boundaries for discontinuity and assignment error.
9. Run fair lending analysis per segment.
10. Set a review that can merge a segment back.

Steps two and six do most of the work. The interaction test is cheap, precedes any build, and answers the question that the segmentation debate is actually about — and in our experience it returns "add a variable" considerably more often than teams expect.

Frequently asked questions

---

This guide is educational and does not constitute legal or compliance advice. Worked figures are stylized illustrations. Model risk management and fair lending obligations apply to segmentation design as fully as to model specification; consult qualified counsel and your model risk function.