When an Input Fails: Fallback Behaviour as Credit Policy

A decision engine calls several data sources and one stops responding. What happens next is usually not a decision anyone made: it is whatever the error handler does, chosen by whoever wrote that function. On 6,200 monthly applications a nine-hour outage is about 78 decisions made under a policy nobody approved, and of the three available behaviours only failing open leaves no trace at all. Fallback behaviour is credit policy, and it is usually written by accident.

The behaviour nobody specified

A decision engine calls several data sources. One of them stops responding. What happens next is, in most institutions, not a decision anyone made.

It is whatever the code does when a call times out, which is usually one of three things chosen by whoever wrote that function: an exception that fails the application, a default value substituted silently, or a retry that eventually gives up.

Our position is that fallback behaviour is credit policy, and leaving it to error-handling defaults means a material part of your credit policy was written by an engineer solving a different problem.

Per our governance analysis, a decision made without a required input is a decision made under a different policy than the approved one, and nothing in a normal approval process covers which policy that is.

What an outage actually costs

Take 6,200 monthly applications and a bureau attribute feed unavailable for nine hours. Applications arriving in that window: about 78.

Three possible behaviours, three different costs:

Behaviour	Consequence
Fail closed: decline everything	78 declines of applicants who were mostly approvable, plus adverse action notices stating a reason that is not about them
Fail open: decide on remaining inputs	78 decisions at unknown risk, booked, with no flag indicating they were made differently
Queue and retry	78 delayed, and at 22% abandonment on delay, 17 lost regardless

None of the three options is wrong in general and all three are wrong in some cases. Per our threshold analysis, which one is right depends on the input, the product, and the size of the exposure, which means the answer has to be configured per input rather than chosen once for the system.

Failing open is the most dangerous of the three because it is the only one that is invisible. A decline is recorded as a decline and a delay is recorded as a delay. An approval made without a key input looks exactly like an approval made with it, and per our decommissioning analysis the question "which inputs were present when this decision was made" is unanswerable unless somebody decided in advance to record it.

Degraded is not the same as down

Complete outages are the easy case because they are obvious. The expensive cases are the partial ones.

• The feed responds and returns nulls. A successful call with empty data, which code frequently treats as a valid answer meaning zero.
• Stale data served from a cache. Technically available, materially wrong, and per our accuracy analysis indistinguishable from current data.
• Partial coverage. The feed works for most applicants and not for a subset, which is the worst case because it is both invisible and non-random.
• Latency without failure. Responses arrive, slowly, so the system works and conversion falls, per our latency analysis.
• A definition change. Per our attribute analysis, an attribute that starts meaning something slightly different is the hardest of all to detect, because every call succeeds.

The first bullet is the one that produces wrong decisions rather than no decisions. An attribute returning null that the model treats as zero is a model receiving a confident, specific, wrong value. An applicant with twelve months of history and a null response is scored as an applicant with no history, and nothing fails.

The third bullet is the one with a fair treatment dimension. If a data source fails selectively, for a particular file type, geography, or record format, then the degraded policy applies to a non-random subset of applicants, and per our proxy analysis that is a disparity arising from infrastructure rather than from any modelling choice.

Designing the fallback per input

The right configuration depends on what the input does, and the questions are the same for each.

1. Is the input decisive or contributory? A missing identity verification is not comparable to a missing one of forty behavioural attributes.
2. Is there a defensible substitute? A second source, a cached value with a stated maximum age, or a model variant trained without that input.
3. What is the exposure? Per our exposure analysis, a small limit tolerates a degraded decision in a way a large loan does not.
4. Can the decision wait? If the product is not time-sensitive, queueing costs conversion and nothing else.
5. Can the applicant supply it? Per our verification analysis, an input unavailable from a feed is sometimes available from the applicant directly.
6. What does the decline reason say? Per our adverse action analysis, a decline caused by your outage needs a reason that is accurate, and "insufficient information" said to someone whose information exists is not.

Item two is where the real engineering investment belongs. A model variant trained without a given input, validated and held ready, converts an outage from a policy question into a routing question. It costs a build and it is the only answer that preserves both availability and defensibility.

Item six is the one most likely to cause a problem later. An adverse action notice is a statement about the applicant, and issuing one that attributes a decline to the applicant's file when the cause was your own infrastructure is inaccurate in a way that is documented and dated.

Detecting it at all

1. Alert on response distribution, not just on errors. A feed returning 100% nulls is returning 200 OK, so error-rate monitoring sees nothing wrong.
2. Alert on fill rate per attribute. The share of applications where each attribute arrived populated, compared against its normal level.
3. Alert on score distribution shift, per our monitoring analysis, which catches degradation that no infrastructure monitor can see.
4. Track approval rate hourly, not daily, because a nine-hour outage is invisible in a daily number.
5. Timestamp and record which inputs were present on every decision, which is the one thing that makes any of this reconstructable.
6. Age every cached value and record the age used in the decision.

Item five is the single most important item in this article. Per our audit trail analysis, an institution that records which inputs were present can answer every subsequent question about an outage: how many decisions were affected, which ones, what the alternative would have been, and whether the affected population was representative. One that does not record it cannot answer any of them, and the information is not recoverable afterward.

Item two is the cheapest detection available. A per-attribute fill rate compared against baseline catches nulls, partial coverage, and definition changes, which are the three failures that pass every infrastructure check.

What to do afterward

1. Identify every decision made during the window, which requires item five above to have been in place.
2. Re-decide the declines. Per our reconsideration analysis, an applicant declined because your feed was down should be reconsidered without being asked to reapply.
3. Flag the approvals as a cohort and monitor them separately, since they are a different population from normally decided accounts.
4. Check whether the affected population was representative, because a selective failure produces a non-random affected group.
5. Correct any adverse action notice whose stated reason was not accurate.
6. Record the incident with its cause, per our institutional memory analysis, because the next person configuring that fallback will otherwise repeat the reasoning that produced it.

Item two is the obligation most often missed and the cheapest to meet. Seventy-eight applicants declined for a reason that was about your infrastructure can be re-decided automatically, in a batch, within hours, and almost nobody does it because the decline closed the case and nothing reopens it.

Item three turns an incident into evidence. A cohort of accounts approved without a key input, tracked separately, tells you what that input was actually contributing, which is information an ordinary monitoring programme cannot produce because the input is always present.

Frequently asked questions

---

This guide is educational and does not constitute legal, compliance, or model risk advice. All figures are stylized illustrations using assumed application volumes, outage durations, and abandonment rates, and do not describe any particular lender or data provider. Adverse action notice requirements, permissible reasons, record retention obligations, model risk management expectations, and fair lending expectations regarding degraded or inconsistent decisioning vary by institution type and jurisdiction. Consult your compliance, legal, and model risk functions before configuring fallback behaviour or acting after an incident.