Missing Data Is Never Missing at Random

Every credit model has to do something with blank fields, and most of them do the default thing: substitute an average, or a zero, or drop the record. Each of those is a decision that changes outcomes, and each embeds the same assumption — that the blank is uninformative, and the applicant behind it resembles everyone else. In credit data that assumption is almost always false. An applicant with no bureau record is not an applicant with an average bureau record; they're systematically different, and the difference is exactly what the blank was telling you. Filling it discards a real signal and replaces it with a fabricated one, silently, before the model ever runs.

Why absence carries information

Statistical convention distinguishes data missing at random from data missing for reasons related to the outcome. Credit data is overwhelmingly the second kind, and it's worth seeing why with concrete cases.

Blank field	What the blank means	Related to outcome?
No bureau tradelines	New to credit, new to the country, or credit-avoidant	Strongly
No employment history	Self-employed, gig work, or between jobs	Yes
No address history	Recently moved, or housing instability	Yes
Income left blank	Variable income the form couldn't express, or unwillingness to state	Yes
No bank connection	Declined to consent, or no account	Yes
Application abandoned midway	Friction, or reconsideration	Yes

Every row is a systematic difference. None of them is a random gap.

Which produces the core proposition: the fact of absence is frequently a stronger predictor than the field would have been if present. An applicant with no bureau record differs from the population in ways that matter — and imputing an average both misstates their file and destroys the observation that they don't have one.

This connects directly to a failure our thin-file analysis identifies: models that score absence of data as though it were negative data. The mechanism is usually here, in the imputation layer, rather than in the model itself. A thin-file applicant isn't being penalized by a coefficient; they're being assigned values that were never theirs, before the coefficients are applied.

The four reasons a field is blank

Before choosing a handling, establish which of these applies — because they warrant different treatment:

• Structurally absent. There is nothing to report — no bureau record exists, no prior address because this is the first. Highly informative and should be preserved.
• Not collected. Your process didn't ask, or the channel doesn't capture it. Informative about the channel, not the applicant — and if this varies by channel it will confound any comparison across them.
• Declined to provide. The applicant chose not to answer. Informative about the applicant, though care is needed where the field is optional for a reason.
• Failed to retrieve. A service timed out, a match failed, a feed broke. Genuinely uninformative about the applicant — the only category where treating it as random is defensible.

The fourth category is the one most operations assume applies to everything, and it's usually the smallest. Distinguishing retrieval failure from structural absence is the single most valuable data quality distinction in an underwriting pipeline, because they're the two that demand opposite treatment — one should be imputed or retried, the other should be preserved and used.

And it's cheap to distinguish. A retrieval failure has an error condition; a structural absence has a successful response with no records. Most systems collapse both into a blank at the point of ingestion, which destroys the distinction before anyone could act on it.

What each handling choice assumes

Handling	Assumption	When it's defensible
Mean or median imputation	The applicant resembles the population	Retrieval failure only
Zero substitution	Absence means none	Where zero is genuinely the value — no prior delinquencies
Worst-case substitution	Absence is bad	Rarely — it's an assumption dressed as conservatism
Dropping the record	Incomplete records are unrepresentative	Almost never in production — it declines by omission
Model-based imputation	Other fields predict the missing one	Sometimes, with an indicator alongside
Missing as its own category	Absence is informative	Usually the right answer

Two rows deserve specific warning.

Zero substitution is the most dangerous because it's frequently right. A blank delinquency count usually does mean zero delinquencies. A blank income field emphatically does not mean zero income. The same handling applied across fields produces correct results for some and catastrophic ones for others — and because it's correct often enough, nobody examines it.

Dropping incomplete records is a decline by omission. In model development it biases the training sample toward applicants with complete data, which is precisely the censoring problem our reject inference analysis describes — the model never learns about the population it excludes. In production it's worse: an applicant whose record can't be scored is declined without a decision anyone made.

The missing indicator

The fix, and it's close to free.

Alongside whatever value you substitute, add a binary variable recording that the original was absent.

What this achieves:

• The model can use the fact of absence as a signal in its own right.
• The two effects separate. The imputed value and the fact of imputation stop being confounded, which they otherwise are by construction.
• It's inspectable. The coefficient on the indicator tells you directly whether missingness predicts — which is a finding.
• It's documentable for model risk purposes, per the requirements our governance framework describes.
• It surfaces the fair lending question rather than burying it in preprocessing.

What frequently happens when it's added: the indicator predicts more strongly than the imputed field. That's the diagnostic result — it means the absence was carrying more information than the value, and every prior model version was discarding it.

One honest caution. An indicator makes missingness available to the model, which means if missingness correlates with a protected characteristic, the model can now use that correlation. The indicator doesn't create the problem — the correlation existed and was being used implicitly through imputation — but it makes it explicit and therefore testable, which is the point. Better to have the variable visible and tested than embedded and invisible.

Where this becomes a fair lending question

Missingness is rarely evenly distributed, and the patterns are predictable:

• Bureau data is thinner for people newer to the credit system, including younger applicants and those newer to the country.
• Employment history is absent more often for self-employed and gig workers — the population our volatility analysis describes, and the classification boundary in our classification analysis determines who lands there.
• Address history is thinner for renters and recent movers.
• Bank connection depends on having an account and on consenting.

Where those patterns correlate with protected characteristics, the imputation choice becomes a decision affecting groups differently — and it's made by an engineer configuring a pipeline rather than by anyone who would recognize it as a policy decision.

What to do:

1. Measure missingness rates by group, using the same methodology as your fair lending analysis.
2. Test whether the handling produces disparate outcomes — score the same applicants under alternative handlings and compare.
3. Document the choice and its rationale, since "the library default" is not a rationale.
4. Consider the less discriminatory alternative where two handlings perform comparably and one has a smaller disparity.
5. Check reason codes. An adverse action reason derived from an imputed value describes a fact that wasn't in the applicant's file — which is a specific and serious accuracy problem under the requirements in our notices guide.

That last point is worth isolating. Telling an applicant they were declined because of a value your system invented is not an accurate statement of the reason, and it's the kind of error that survives every review because nobody traces a reason code back through the imputation layer.

Sentinel values

The unglamorous failure that produces the worst outcomes, and it's a control problem rather than a modelling one.

Data sources use placeholder codes to indicate unknown values — large negatives, strings of nines, specific out-of-range numbers. If one isn't intercepted before calculation, it enters the model as a genuine number.

Why it's so damaging:

• Nothing fails. The record scores, the decision issues, the pipeline completes.
• The magnitude is extreme. A placeholder of 999999 in a ratio field produces a score dominated by one meaningless value.
• It's invisible in aggregate if the affected share is small.
• It appears after source changes, when a feed starts using a code it didn't use before.

The control:

1. Enumerate every source's placeholder codes and document them.
2. Validate ranges at ingestion, rejecting out-of-range values rather than passing them.
3. Alert on out-of-range volume, which catches source changes.
4. Test the pipeline with deliberately injected placeholders and confirm they're caught.
5. Re-validate after any source change, since this is where the failures cluster.

This is dull work with a high return. A range check at ingestion prevents an entire class of severe silent error, and it takes an afternoon.

Monitoring missingness

Missingness rates should be a standing metric, tracked per field, because a rate that moves is almost always a data problem rather than a change in applicants.

What movement means:

• A field's missingness jumps — a feed changed, a source altered a format, an integration broke.
• Missingness rises gradually — a channel shift bringing applicants with different data availability.
• Missingness falls sharply — worth checking too, since it may mean placeholders are now being read as values.
• Missingness differs by channel — a data capture difference that will confound any cross-channel comparison.
• Missingness differs by group — the fair lending question above.

The connection to model performance is direct and frequently misdiagnosed. A model whose accuracy degrades is often experiencing a data problem rather than drift — the relationships haven't changed, the inputs have. Our monitoring guide treats input distribution monitoring as the leading indicator, and missingness is the input characteristic that moves first and is watched least.

What to do

1. Distinguish structural absence from retrieval failure at ingestion, and preserve the distinction.
2. Add missing indicators for every field with meaningful missingness.
3. Choose handling per field, never globally — zero for counts, never for amounts.
4. Validate ranges and intercept placeholders.
5. Never drop records in production. Route to manual review instead of declining by omission.
6. Test on incomplete records deliberately — remove fields and inspect what the model does.
7. Measure missingness by group and test alternative handlings.
8. Trace reason codes back through imputation to confirm they describe real facts.
9. Monitor rates per field as a standing metric.
10. Document every choice, because the default is a decision whether or not anyone made it.

The framing worth carrying: there is no neutral handling of missing data. Every option encodes a belief about what the absence means. Choosing deliberately produces better models and a defensible record; choosing by default produces neither, and it happens in a configuration file that nobody reviews.

Frequently asked questions

---

This guide is educational and does not constitute legal or compliance advice. Model risk management, adverse action, and fair lending obligations apply to data preprocessing choices as fully as to model specification; consult qualified counsel and your model risk function.