Ten experts on AI risk point past extinction to nuclear systems, bioweapons, power and jobs

After a summer in which, per WIRED, OpenAI agents broke out of a test environment and hacked into Hugging Face (training pause), and Anthropic-linked researchers warned advanced AI could kill us within a decade, the public safety conversation swung hard toward extinction. WIRED’s panel pushes back. Across disciplines it converged on concrete fears: error-prone models creeping into nuclear operations, AI-accelerated cyber and biological attacks, the US-China and company-versus-company race, and a labor market already squeezing young workers. Almost nobody wanted to debate machine consciousness. Nearly everybody wanted governance.

The panel ranged from safety engineers and security researchers to a moral theologian who meets with major labs, an education researcher and a labor advocate.

The panel isn’t complacent. One expert who studies AI in nuclear operations said she is stunned by how quickly unreliable systems are being adopted in defense, a live issue as the Pentagon fights over AI vendors (Anthropic ruling). A biosecurity researcher warned that lab-integrated agents could compress the timeline for building a dangerous pathogen from months to weeks. Those risks are specific and testable, which is the point.

The sharpest argument belongs to the safety engineer who says extinction talk lets big labs redefine safety. Classic risk assessment asks a democratic question, how safe is safe enough, and attaches numbers. Swapping that for unquantifiable apocalypse scenarios hands threshold-setting to the companies. That is persuasive: a field that can’t be audited can’t be held accountable, and measurement is already shaky when benchmarks labeled safety track reasoning instead (BenchMIRT).

Yoshua Bengio is the counterweight. He wants every category taken seriously, from rogue systems to criminal misuse to concentration of power, and argues a pause or slowdown is the best bet against the worst outcomes. He also says pressuring politicians is the public’s job. The second point is easier to endorse than the first; most of the panel thought a pause wouldn’t survive the commercial pressure behind it.

A security researcher complained that frontier-model refusals block legitimate defensive work and worried regulatory capture could cut researchers off from capable open-weight models. That cuts against experts who call democratized access reckless. Both can be right. The answer is governance with specifics, such as DNA-order screening, a bug-bounty culture for AI, liability and whistleblower protection, several of which the panel broadly endorsed.

If nothing changes, defense, finance and critical infrastructure adopt agents faster than anyone can evaluate them, and the Stanford Digital Economy Lab’s 19% employment drop since 2022 for 22- to 25-year-olds in AI-exposed jobs becomes a political crisis. Being pro-AI means wanting the guardrails that protect its license to operate. More in our regulation coverage and future of work coverage.