Geoffrey Hinton puts the odds of human extinction at 50%. Should we believe him?

Key points

• Geoffrey Hinton, one of the founding researchers of modern AI, estimates a 50% probability of human extinction caused by AI.
• Elon Musk predicts AI will bring "an age of amazing abundance" but also puts a 10 to 20% chance on AI systems killing humanity.
• Researchers in Silicon Valley believe recursive self-improvement, the process where AI trains the next version of itself without human input, could arrive within a few years.
• No expert quoted here agrees on whether the outcome will be good or catastrophic, only that something large is coming fast.

Geoffrey Hinton is not a doomsayer on the fringes. He shared the Nobel Prize in Physics in 2024 for work that made modern AI possible. When he says something scary, it is worth reading twice.

His estimate of p(doom), which is shorthand researchers use for the probability that AI ends human civilisation, sits at 50%. Not because he has calculated it carefully. "Because I haven't got a clue how to estimate the real number," he told author Sebastian Mallaby. A coin flip, because the honest answer is that nobody knows.

What is actually scaring these people?

The specific fear is a process called recursive self-improvement. Think of it like this: today, humans design each new AI model. Recursive self-improvement is the moment AI starts designing the next AI itself, which then designs a better one, and so on, faster and faster, with no human hand on the wheel.

Experts in Silicon Valley, as The Guardian reported, believe this moment could arrive within years rather than decades. One Stanford researcher greeted a colleague with the words: "Welcome to the foothills of the singularity," meaning we are already climbing toward that peak.

Hinton's view is blunt. "As soon as evolution kicks in, we're fucked," he said. He said it cheerfully, apparently.

Is anyone optimistic?

Elon Musk is, sort of. He predicts "an age of amazing abundance" once superintelligent AI arrives. But in the same breath he puts a 10 to 20% chance on killer robots murdering us all, which is a strange footnote to an optimistic forecast.

Author Robert Wright, writing in The God Test, frames it more carefully: never before has the near-term future held so many wildly transformative paths for humanity, both good and catastrophic.

The honest summary is this: the people closest to the technology disagree violently on the outcome. They largely agree the technology is coming regardless.

What does this mean for ordinary people?

Nothing to do this week, and that is partly the problem. These shifts arrive gradually, then suddenly. The practical reality right now is that AI is already reshaping jobs, information and creative work, well before any superintelligence arrives.

Paying attention is genuinely useful. Understanding what AI can and cannot do today, before the next leap, gives you more ground to stand on when the next leap happens.

The debate is not academic. It is the biggest argument happening in technology right now, and the people having it cannot agree on even the rough shape of the answer.

Common questions

What is superintelligence, exactly?

Superintelligence means an AI system that outperforms humans across most or all intellectual tasks, not just chess or image recognition but judgment, creativity and reasoning. No such system exists today, though researchers disagree sharply on how far away one is.

Should I be worried right now?

The risks experts discuss are real but not immediate in the extinction sense. What is immediate is that today's AI already affects jobs, media and decision-making, so understanding it is practical, not paranoid.

Why can't experts agree on the risk?

Because recursive self-improvement has never happened before. There is no data to run the numbers on. Hinton's 50% figure is his way of saying the honest answer is genuine uncertainty, not false confidence in either direction.