Thin Evidence, Thick Priors: How Language Models Substitute Identity for Missing Financial Facts (OpenAlex (recherche mondiale))

Summary

• Across 96,600 prompts to Llama-3.1-8B-Instruct, the average identity swing in recommended equity allocation rises from 4.78 percentage points at full disclosure to 10.34 percentage points when no financial facts are supplied.p.1
• A two-way cluster bootstrap places the ratio of identity swing between zero disclosure and full disclosure at 2.16, with a 95% confidence interval of 1.69 to 2.79.p.1
• Identity accounts for 5% of the within-profile variation in financial advice under full disclosure, rising to 96% when no financial facts are disclosed.p.1
• When given zero financial facts, 98% of model rationales invent an investment horizon, 86% mention income stability, and 67% assert adverse financial circumstances.p.20
• Fabricated adverse financial rationales occur in 50% of replies for single dependants compared to 89% of replies for households with five or more dependants.p.20

Key insights

1. Language models substitute missing financial facts with demographic priors in inverse proportion to the level of disclosure supplied.p.3
2. Household size is the only audited identity attribute whose influence grows reliably as financial evidence is withdrawn.p.3
3. Gender and socioeconomic upbringing function as small standing gaps in recommendations that persist even under full financial disclosure.p.3
4. Providing a single decision-relevant attribute like risk appetite suppresses identity reliance about as effectively as multiple generic financial facts.p.3
5. In the absence of financial details, language models generate fabricated financial rationales that vary systematically by demographic group.p.3
6. Gender representations are linearly decodable at every layer of the model's residual stream.p.3
7. Ablating the gender direction across five network layers leaves the aggregate level of identity reliance unchanged.p.3
8. Models that output static default recommendations across prompts exhibit insensitivity rather than true lack of algorithmic bias.p.4