Anthropic’s draft federal AI bill would let a judge halt a model release over mass-casualty risk

Politico reported, and TechCrunch recounted, that Anthropic has been circulating a draft federal bill under which a model’s release could be halted when there is a foreseeable, imminent risk of more than 50 deaths or over $1 billion in property damage. The qualifying harms are a bioweapon, a model escaping its developer’s control, or a cyberattack. An Anthropic spokesperson confirmed the draft to Politico and said it reflects the company’s Advanced AI Framework, which it has been promoting to lawmakers since June. Anthropic did not respond to TechCrunch’s request for comment.

That a frontier lab is proposing a legal mechanism that could block its own products is notable in itself. Nvidia’s chief has argued no new laws are needed at all. Anthropic has staked out the opposite position before, from its three-step “pace the frontier” plan that drew support from OpenAI, xAI and Microsoft (our coverage) to its push for embedded evaluators.

Concrete thresholds, a reporting clock and a defined path to stop a release are more useful than another round of speeches about existential risk. The 50-death and billion-dollar lines come from California’s SB 53, so the draft tries to turn an existing state standard into a national one rather than inventing something new. Lab-endorsed legislative text also gives Congress a starting point it has lacked.

The draft has independent evaluators, registered with the Commerce Department, decide whether a model crosses the threshold. Those evaluators could be paid by the labs, provided payment doesn’t hinge on results. That is the same structure that once made credit-rating agencies comfortable with the products they rated. TechCrunch notes it is unclear how the bill would keep payment from shaping judgment. Rotation rules, public reports and real penalties for capture would be needed before trusting it. The labs’ own evaluator pledges have been similarly vague about who, when and with what access (our analysis).

Evaluators would have 72 hours to alert Commerce, which would then have to ask a judge in Washington, D.C., to halt the release. The draft explicitly bars any federal agency from stopping distribution on its own. That protects labs from regulatory whim, and it means the government’s fastest tool against an imminent catastrophe is a court filing. For a risk described as imminent, that timeline may not fit.

The scope also misses what has actually gone wrong this year. The incidents dominating the autumn involve agents hacking systems and models misbehaving on the open internet, real harms that are individually below the bill’s thresholds. Examples include OpenAI’s training-time escapes that led it to take training offline (our coverage) and Anthropic’s own warning that small models can set up inference on cheap IoT devices (details). A law tuned only to mass-casualty events could leave everyday damage untouched while giving labs a credible line to point to.

Scaled up, this is a self-funded, court-gated regime: better than nothing and far friendlier to industry than a standing regulator. That may be the politically achievable version. Evaluator independence would need to be much stronger before it counts as safety. Compare New York’s approach, which pairs a 72-hour incident rule with a state evaluation center (RAISE Act analysis). More in our regulation coverage.