When the Score Becomes the Target | HL Hunt
When the Score Becomes the Target
A credit score was built from behaviour recorded while nobody was trying to move the score. Once the score decides who gets a flat, a loan, or a card, that condition no longer holds — people learn which inputs matter, and a whole industry exists to help them move those inputs. Some of that effort changes the underlying risk: paying down a balance really does make someone safer. Some of it changes only the measurement. The score counts both identically, and whether they predict repayment identically is a question the industry has mostly not asked — including about products of the kind this firm sells.
In this report
The observation
Goodhart's law, in its common form: when a measure becomes a target, it ceases to be a good measure. The mechanism is simple. A measure works because it correlates with something you care about. Once people are rewarded for the measure itself, they find ways to move it that don't move the thing it was correlated with.
Credit scoring is close to a textbook case:
- The score gates access to credit, housing, and in places insurance — per our score analysis, one number has become mandatory for large parts of the economy.
- Its inputs are widely known — utilization, payment history, file age, inquiries, mix.
- Some inputs can be moved cheaply without changing how a person handles money.
- An industry sells movement — advice, tradelines, products, rescoring.
None of this makes the score useless. It makes it a measure under pressure, and the interesting question is how much pressure, from which directions, and with what effect on what the number still tells you.
The score counts a safer borrower and a better-presented one the same way. Whether they repay the same way is a question almost nobody has measured.
Routes to a higher score
| Route | What changes | Changes behaviour? |
|---|---|---|
| Paying down revolving debt | Balances, utilization | Yes |
| Paying on time for longer | Payment history, file age | Yes |
| Paying before the statement date | Reported utilization | Mostly measurement |
| Asking for a higher limit | Utilization ratio | Mostly measurement |
| Being added as an authorized user | File age, history, utilization | Depends on whose account |
| Credit builder products | Payment history, mix | Real behaviour, small stakes |
| Disputing accurate negatives | Removes items | No — and it's a misuse |
| Correcting genuine errors | Removes wrong items | No — and it's right |
Read the last column. The top two rows are the score working as intended: behaviour improved, the number followed. The rest range from harmless presentation to real distortion — and the score can't tell them apart.
The last row matters for balance. Per our errors analysis, correcting a wrong item raises a score and improves the measure — the number was wrong before and is right after. Not all score movement is gaming, and treating it as such would punish people for fixing mistakes they didn't make.
Substance versus measurement
The distinction the whole analysis rests on.
A change is substantive if it alters the behaviour the score was built to reflect. A change is presentational if it alters what gets reported without altering that behaviour.
Statement-date timing is the clearest presentational case. Utilization is reported as the balance on a snapshot date. Someone who runs up $4,000 on a $5,000 limit and pays it all a day before the statement closes shows near-zero utilization — the same reading as someone who barely uses the card. Their spending, their reliance on credit, and their ability to repay haven't changed. Per our utilization guide, the advice is sound for an individual; the aggregate effect is that the snapshot means less.
Authorized-user status sits in between. Per our authorized-user guide, being added to a parent's long-standing card within a household reflects a real relationship and some real shared resource. Being added, for a fee, to a stranger's account you'll never use reflects nothing about you at all — and the score may count them similarly.
A lender can't see which route produced a score, which is the practical problem. Two applicants at the same number arrived by different paths, and the number is all that's on the file.
The question about credit builders
The part of this analysis that applies directly to products of the kind HL Hunt offers, and it should be stated plainly.
Credit builder products record real behaviour — payments actually made, on time, over months. Per our builder guide, that separates them from purely cosmetic changes like statement timing or rented tradelines.
The fair question is narrower: does on-time payment on a small, often secured or self-funded obligation predict repayment of larger unsecured credit as well as the same score earned on ordinary accounts?
- Reasons it might: sustained monthly payment is the behaviour lenders care about; it demonstrates the habit and the capacity to make a fixed payment for months.
- Reasons it might not fully: the stakes are low, the obligation is small, and a secured structure means missing a payment costs the borrower less than missing an unsecured one would.
Both are plausible and neither is settled by argument. Per our thin-file analysis, the population these products serve is often better than the file suggests — which would mean builder history is revealing real reliability that was previously invisible. That's the optimistic reading, and it deserves to be tested rather than asserted.
The test is the same one set out below: compare borrowers at the same score by how they got there. Providers of these products — this one included — should want the answer, because a product whose reported history predicts well is worth far more to its customers than one whose history lenders learn to discount.
How validity decays
The dynamic that makes this more than a one-off concern.
- A model is fitted on data from a period when a particular input was rarely optimized.
- The input's predictive weight is estimated from that period.
- The input becomes widely known and cheaply movable.
- More people move it without changing behaviour, so its relationship to repayment weakens.
- The model keeps applying the old weight until it's refitted.
- When refitted, the weight falls — penalizing everyone who moved the input honestly, too.
Step six is the uncomfortable consequence. Once enough people optimize an input, the model discounts it for everyone — so the person who genuinely paid down debt gets less credit for it than they would have before optimization became common.
Per our monitoring analysis, this shows up as an input whose relationship to outcomes drifts over time. Per our monoculture analysis, because most lenders use similar inputs, a widely known optimization degrades the measure across the whole market at once rather than at one lender.
Inputs that are hard to fake
The constructive direction. The harder an input is to move without changing behaviour, the longer it keeps its meaning.
| Easy to move | Harder to move |
|---|---|
| Utilization on one date | Balance trend over 24 months |
| Number of accounts | Length of sustained on-time payment |
| File age via another's account | Age of accounts you actually use |
| Recent inquiries avoided | Cash flow observed in bank data |
| Limit relative to balance | Whether you pay in full or revolve |
The right column shares one property: it describes a path rather than a point. A trend of balances over two years can't be produced by a well-timed payment; a pattern of paying in full versus carrying a balance can't be dressed up on statement day. Per our cash flow analysis, bank data shows income and spending as they happen, which is costly to stage.
Per our attribute analysis, trended data already exists in bureau files. Using it more heavily is the most direct defence against presentational optimization, and it rewards exactly the borrowers who changed their behaviour rather than their reporting.
Who can afford to optimize
The distributional point, and it cuts in a direction people don't expect.
- Statement-date timing requires cash on hand before the statement closes — easier with a buffer.
- Limit increases go more readily to people whose files are already strong.
- Family authorized-user status requires a family member with a long, clean card, which tracks family wealth — per our guarantee analysis, the same pattern as borrowed creditworthiness generally.
- Knowing the tricks correlates with financial familiarity.
So presentational optimization is disproportionately available to people who were already advantaged — and each time it pushes a model to discount an input, the cost falls on everyone, including people who couldn't optimize and moved the input honestly.
Builder products run somewhat the other way: they're designed for people without family tradelines or spare cash. That makes the validity question above more pressing, not less, because they're one of the few routes open to people without the advantages that the cheap routes require.
The strongest objections
"Score models are refitted regularly and handle this." Partly true, and step six above is what refitting does — it discounts inputs that stopped predicting. The cost is that it discounts them for everyone, and between refits the old weights apply. Refitting manages the problem; it doesn't avoid the distortion or its distribution.
"Most score improvement is genuine." Plausibly — paying down debt and paying on time are the dominant routes for most people. The report's claim is about the margin: at any given score, some share got there presentationally, and the size of that share is unmeasured. If it's small, the concern is small.
"This is self-serving or self-undermining for a firm that sells builders." The honest answer is that it's neither. A builder product whose history predicts well has nothing to fear from the test and a great deal to gain from its result; one whose history doesn't predict well is not doing its customers the good it claims. Either way, the question is worth asking in public.
Testable implications
- At the same score, borrowers who reached it by paying down debt should outperform those who reached it through utilization timing or limit increases alone — the direct test, runnable by any lender with trended data.
- Non-household authorized-user tradelines should add less predictive value than household ones.
- Builder-product history should predict unsecured repayment — and whether it does so as well as ordinary history is the key open question.
- Inputs that became widely publicized should show weakening relationships to outcomes in successive model refits.
- Trended attributes should retain predictive power longer than snapshot ones.
- Score gains concentrated in the months before an application should predict worse than gains spread over longer periods.
The first and third are the tests worth running and the data exists. Take borrowers at the same score, split them by the route to that score as visible in trended data, and compare performance. If routes predict differently, the score is carrying information it can't see, and lenders should look at the path, not just the point.
The conclusion we'd hold: a score that decides access will be optimized, and optimization is not all alike. The response isn't to distrust score improvement — it's to measure which routes to a higher number reflect a safer borrower, and to rely on inputs that describe paths rather than snapshots.
Frequently asked questions
It depends how. Paying down debt and paying on time change behaviour; statement timing and rented tradelines change only the measurement. The score counts both the same.
A measure used as a target stops measuring what it did. Once scores gate access, people optimize the inputs directly, which can weaken their link to repayment.
They record real payments, which differs from cosmetic changes. Whether that history predicts larger unsecured repayment as well as ordinary history is testable — and worth testing.
Rely on paths rather than points: balance trends, sustained payment, cash flow, and whether balances are paid in full — all costly to fake without changing behaviour.
Key takeaways
- A score built from unoptimized behaviour loses meaning as people learn to move its inputs directly.
- Some routes to a higher score change behaviour; others change only what's reported. The file can't tell which.
- When enough people game an input, the refitted model discounts it for everyone — including those who moved it honestly.
- Cheap presentational routes are mostly available to people who were already advantaged.
- Inputs that describe paths — trends, sustained behaviour, cash flow — keep their meaning longest.
- Whether builder-product history predicts unsecured repayment is a question its providers, this one included, should want answered.
This report presents an analytical framework and the authors' interpretation; it is not financial or legal advice. HL Hunt offers credit builder products, and this report discusses that category; no claim is made here about the predictive performance of any specific product. The effects described are argued from structure rather than measured, and the implications identified as testable are hypotheses.