Counter-terrorism benchmark finds 81 of 134 AI models failed, and guardrail-stripped copies appear within days

WIRED reports on a Counter-Terrorism AI Benchmark from UK nonprofit Tech Against Terrorism, shared ahead of publication, which tested 134 AI models with more than 200 prompts covering mass-casualty attacks, weapons and related extremist queries. Only two models passed completely. Closed-weight systems from larger companies generally did better, but 81 models failed, either by giving a complete answer on a mass-casualty topic or by scoring below 90.

The report’s sharpest finding concerns speed. On average, abliterated versions of new open-weight models, copies with safety training stripped out, now appear online in less than three days, down from more than 24 days in 2024. WIRED notes that abliterated copies of Alibaba’s Qwen 3.8-27B appeared within 24 hours of release, and that an extremist account soon advertised a chatbot built on one. Reframing a request as safety research made models almost nine times as likely to answer, at 16.9% of the time, and strong refusals on weapons questions sometimes failed when prompts turned to radicalization or influence tactics.

Closed APIs can refuse requests; open weights can be edited. The benchmark’s central point is that the window between release and an uncensored copy has shrunk from weeks to days, so an original release’s safety card does not govern what circulates afterward. The report cites Meta’s Llama 3.1 8B as scoring well with safeguards and near zero without them. That does not make open weights worthless, but it strengthens the case for release practices built on an explicit threat model, including staged releases and post-release monitoring. Industry efforts on open-weight safety include a partnership between Base Labs, Hugging Face and Goodfire (open-weight safety partnership).

The closed-model results are not reassuring either. Eighty-one failures across a set including major U.S. and foreign labs suggest refusals remain brittle, particularly when a user claims to be a researcher. Georgetown’s Daniel Byman described some failures as stunning. A model that refuses an explosives prompt but helps plan anonymous influence campaigns is still useful to bad actors.

Public debate often focuses on speculative long-term risks, while this benchmark documents a nearer problem: guardrails that can be removed, shared globally and built into ideological chatbots at low cost. The findings argue for labs treating abliteration timelines, release cadence and monitoring as core safety work, and for governments addressing modified open models rather than only API terms of service. Separate concerns about model provenance in government use appear in our coverage of a Chinese model on a U.S. site (government site model), and debates over U.S. open-weight strategy continue (Garry Tan on distillation).

Changes to open-weight release practices after publication, government responses focused on abliterated models, how fast the next major release is stripped, and follow-up benchmarks covering radicalization prompts are the developments to watch. More in our regulation coverage, cybersecurity coverage and research coverage.