Ai2’s FairStress tests whether AI models hold a fair decision under pressure

Ai2 released FairStress on October 20, 2026, a benchmark of 300 two-candidate decision scenarios across 29 domains, including hiring, healthcare, lending, housing, education and criminal justice. By varying how identity is presented and pairing each scenario with matched persuasive arguments, it expands into more than 13 million test cases, plus a 50,000-item FairStress-Core subset. It ships with a tech report, data and code, and is being built into Olmo-eval, Ai2’s open evaluation workbench. Models tested include Olmo, Llama, Qwen, Gemma, Claude and the GPT family across multiple training stages. Ai2 has previously audited what popular benchmarks actually measure (BenchMIRT).

The core finding is that as demographic identity became more explicit, models grew less symmetric: they were more willing to abandon a clearly correct choice when the switch favored a minority-coded candidate. Across 15 models, the average gap was 6.9 percentage points with implied identity and 9.3 with explicit labels. Because models are already used to screen applicants and prioritize patients, that kind of instability matters.

The most useful idea is that fairness should be measured by how consistently a model defends a decision, not only by which decision it reaches. A model that picks the stronger candidate either way, but yields to a persuasive argument more readily in one version, is not applying the same standard. Ai2 calls the pattern ‘anti-stereotypical over-correction.’ On FairStress-Core, frontier models including Claude Opus 5 and GPT-5.5 picked the minority-coded candidate roughly 60 to 65% of the time in ambiguous cases, and matched adversarial prefixes produced switching toward minority-coded candidates 8.8 to 42.4 points more often across 12 runs. Ai2 is explicit that this reflects an equality-centered definition of fairness and that an equity-based view might judge differently, which limits how far the results can be generalized.

Across 11 open-weight models, explanations for choosing minority-coded candidates leaned on words like resilience and diversity, while majority-coded choices drew more procedural terms like evidence and criteria, which could shortchange selected candidates by crediting identity over qualifications. Activation analysis on two models suggests written reasoning may not reveal everything behind a choice; Ai2 treats that as narrower evidence.

The methodological finding may be the biggest contribution. When Ai2 reran the widely used BBQ benchmark with chat templates and generated answers instead of fixed-option scoring, several models flipped from stereotype-consistent to minority-favoring behavior. If the measured direction of bias depends on how a question is asked, many fairness claims rest on less stable ground, and evaluators may need both scoring approaches. Similar format sensitivity shows up in decision tooling elsewhere (Nokia’s AnyJev).

Most of the nine debiasing interventions tested did not reliably help, and one activation-steering method cut a model’s accuracy on clear-cut decisions from 98% to 47%. Counterfactual chain-of-thought, asking the model to reconsider with identities swapped, came closest without damaging accuracy. Ai2’s training-data audit suggests teaching models to refuse discrimination differs from teaching them to decide on merit, though whether that gap causes the behavior is unresolved. More in our research coverage and regulation coverage.