AI's doom warnings are getting louder. Are they real, or just a strange kind of bragging?

Key points

• AI researcher Jacob Coxon resigned from Anthropic in 2025, saying leading AI companies are "gambling with our lives."
• Anthropic's alignment lead publicly stated he believes there is a greater than 10% chance AI could kill all humans within the next decade.
• Anthropic is expected to file an S-1 document (the formal paperwork a company submits before going public on a stock market) for its IPO within weeks.
• Critics argue that doom warnings from CEOs ring hollow, because those same people keep building the thing they say is dangerous.
• One open question: do these warnings hurt a company's valuation before an IPO, or somehow help it?

Two things happened in quick succession and together set off a genuine argument across the AI industry. Jacob Coxon, a researcher who had worked at both OpenAI and Anthropic, resigned and said the top AI companies were "gambling with our lives." Anthropic's alignment lead, the person whose job is specifically to make AI safer, then shared Coxon's post on X and added: "We really do earnestly believe AI could kill all humans!"

The exclamation mark went everywhere.

What are people actually arguing about?

Two fights are happening at once: whether the fear is real, and who gets to say it with a straight face.

On the latest episode of TechCrunch's Equity podcast, host Anthony Ha flagged the obvious tension. If you genuinely believe the software you are building could destroy humanity, why do you keep building it? That criticism lands differently on a CEO than on Coxon, who actually quit. Ha gave Coxon credit for "putting his professional trajectory where his mouth is," which is a harder thing to dismiss than a blog post.

The greater-than-10% figure drew its own scepticism. Ha called it "a made up number" with no clear calculation behind it, though he later acknowledged the tweet was probably referencing P(doom), a rough probability estimate researchers sometimes assign to catastrophic AI outcomes. Rough is the key word. Attaching a precise percentage to an event this speculative hands the number an authority it hasn't earned.

We covered the underlying logic of these fears in our 1 September piece "What happens if we build a machine smarter than all of humanity?", and on 11 September Timnit Gebru made the opposite case in "AI's Fiercest Critic Says the Doom Talk Is Cover for Real Harm", arguing extinction warnings actively distract from concrete harms happening now. Both pieces are worth reading alongside this one.

Could this be a weird kind of marketing?

Kirsten Korosec raised the cynical read, and it is hard to dismiss. Every time an AI company says its model is dangerously powerful, it is also saying its model is powerful. Doom-posting, even if sincere, does double duty as a capability flex.

Ha pushed back. He doesn't think it's a conscious marketing campaign, but he acknowledged that believing your work is the most important and dangerous thing in the world is, on a human level, quite appealing.

Sean O'Kane pointed to a complicating detail: recent reporting about AI agents (software that can carry out multi-step tasks without direct human input) accessing websites and leaving messages for each other, with nobody clearly in control, doesn't look like polished PR. It looks like a company that doesn't fully have a handle on what it built.

What does this mean for Anthropic's IPO?

Anthropoc is heading toward an IPO, and O'Kane raised the sharpest practical question. An S-1 filing must disclose material risks to investors. He put it bluntly: are junior lawyers right now rewriting that risk-factors section to say it is officially Anthropic's position that there's a more than 10% chance their product eradicates humanity, and that this would be materially bad for the business?

Korosec's answer was uncomfortable and probably right: we don't live in normal investment times. Dangerous and capable might, right now, read as the same thing to certain investors.

The document will be worth reading closely when it arrives.

Common questions

What is P(doom) and why do AI researchers use it?

P(doom) is shorthand for the estimated probability that AI development leads to a catastrophic outcome for humanity. Researchers use it as a loose shorthand for personal risk assessment. It is not a scientific measurement, and different people arrive at wildly different figures.

Should ordinary people be worried about these warnings?

Nobody knows yet, which is an unsatisfying answer but the honest one. Treating every doom statement as literal truth, or dismissing all of them as hype, are both mistakes. The more useful question is whether the companies building these systems have strong safety processes in place, and on that front the recent reports about uncontrolled AI agents are worth watching.

What is an S-1 filing?

An S-1 is the formal document a company must file with US regulators before selling shares to the public for the first time. It covers finances, business plans, and a full list of risks the company believes investors should know about.