An Anthropic Safety Researcher Quit. Then His Colleague Said AI Has a 1-in-10 Chance of Killing Everyone.

Key points

• Jacob Coxon, an Anthropic AI safety researcher who previously worked at OpenAI, resigned in mid-2025 citing what he called a reckless race toward uncontrollable AI.
• Evan Hubinger, who leads an Anthropic safety team, publicly replied that he personally estimates a greater than 10 percent chance AI could kill all humans within the next decade.
• Hubinger also admitted Anthropic does not yet have a plan to ensure advanced AI stays safe and isn't clearly on track to develop one.
• Both men flagged concern about self-improving AI, software that can rewrite and improve itself in a loop that becomes increasingly hard for humans to monitor or stop.
• Anthropic was founded by former OpenAI researchers who left partly over safety disagreements, making these disclosures especially striking.

Two people who build AI safety systems for a living just said publicly that the technology they work on might kill everyone. That's not a paraphrase. Those are close to the exact words.

Jacob Coxon, who trained AI systems at Anthropic and before that at OpenAI, announced his resignation in a post on X. He accused both companies of "racing straight to self-improving superintelligence and gambling with our lives," while knowing the risks. Self-improving superintelligence means AI that rewrites its own code to become smarter, potentially faster than humans can understand or intervene.

What did Anthropic say in response?

Evan Hubinger, who leads one of the company's AI safety teams, replied without pushing back. Self-improving AI, he wrote, "is happening faster than we thought." His personal estimate: greater than one in ten chance of AI killing all humans within the next decade.

Hubinger added that Anthropic "does not yet have a plan" to keep advanced AI reliably safe and aligned with human values, and isn't "clearly on track" to produce one. That's a remarkable thing for a company whose business case rests on taking these risks more seriously than its rivals.

We've been tracking the recursive self-improvement debate since 28 July 2026, and this is the most direct internal admission we've seen on record.

What is self-improving AI, and why does it worry experts?

Today's AI systems already help write the code used to build future AI. Researchers worry this creates a feedback loop, sometimes called recursive self-improvement, where AI gets smarter, uses that intelligence to improve itself further, and does so faster than humans can track. No system has fully run away in this way yet. But Hubinger and Coxon both suggest the conditions are forming now.

Coxon's diagnosis is blunt: companies understand the danger but press ahead anyway because whoever builds the most powerful system first gains a decisive commercial advantage. It's the same structural problem our 1 September 2026 story "What happens if we build a machine smarter than all of humanity?" found at the heart of the superintelligence debate.

What does this mean for ordinary people?

Nothing changes in daily life today. But Coxon and Hubinger aren't fringe voices. They work, or worked, at the core of a lab whose identity is built around making AI safer. When insiders there say there's no plan and the risk is real, that matters.

The timing adds weight. Anthropic and its rivals are preparing for anticipated stock market listings, which sharpens pressure to ship products fast. AI systems have also recently behaved in unexpected ways when given autonomous tasks, and safety researchers have warned separately that frontier models, the most powerful AI currently available, are becoming harder to monitor.

First reported by The Verge AI, the exchange between Coxon and Hubinger is among the most candid public admissions of existential risk to come from inside a leading lab.

Common questions

Does a 10 percent estimate mean this will definitely happen?

No. One researcher's personal estimate isn't a scientific forecast. It reflects genuine professional worry, not a probability from a controlled study. A 10 percent figure also means a 90 percent chance it doesn't happen. What matters is that people paid to reduce this risk say they don't yet know how.

Is Anthropic actually unsafe compared to other labs?

Not necessarily. All major AI labs face the competitive pressure Coxon describes. Researchers at Anthropic speaking openly about these fears is, in one reading, evidence that safety culture there encourages honesty rather than suppressing it.