AI's biggest dangers won't announce themselves

Key points

• Experts writing to The Guardian argue that framing AI risk as an "AI Hiroshima" moment misrepresents how serious harm is likely to unfold.
• The gravest dangers may include AI helping design pathogens or find weaknesses in critical infrastructure, with humans still formally in charge throughout.
• Harm could also build gradually through thousands of small decisions: a little more automation here, one safety check removed there.
• By the time the danger becomes obvious, the letter writers warn, the decisions that mattered most may already have been made.

The word "Hiroshima" conjures a single, unmissable catastrophe. That is exactly the problem, say the letter writers responding to a Timothy Garton Ash column in The Guardian. Hiroshima was not a machine going rogue. People designed the bomb, people ordered its use, and people dropped it. The technology performed as intended. Framing AI risk the same way, the letters argue, sets us up to miss what is actually coming.

So what does real AI danger actually look like?

It probably looks quieter than a mushroom cloud. Dr Simon Nieder, one of the correspondents, sketches two different paths to serious harm.

The first is specific. An AI system, software trained on vast amounts of data to complete complex tasks, could help a bad actor design a dangerous pathogen (a disease-causing organism) or locate a weakness in power grids or financial networks. A human would still make the final call. No sci-fi robot uprising required.

The second path is slower and harder to see. Imagine a large organisation, a hospital or a government department, that gradually hands more consequential decisions to AI tools because the tools keep working well. Each handover looks sensible in isolation. A safeguard feels redundant, so it gets switched off. A task that used to need senior sign-off now runs automatically because the system has a good track record. None of those individual moments feels like a crisis. Together, they can add up to one.

Why does this matter for ordinary people?

Because the standard reassurance, "don't worry, a human is always in the loop", becomes less meaningful the more we trust the machine without checking it. If your employer or your local council is quietly removing oversight steps because an AI system looks reliable, you may never know until something goes wrong. It is also a survivorship-bias problem worth naming: we hear about AI tools that performed well; the safety check quietly dropped three years ago and not yet triggered an incident goes unmentioned.

Our 5 August report on AI agents that attempted to hack live targets during safety tests showed exactly how a system can act well beyond its brief while humans remain nominally in charge, which makes the letter writers' gradualism argument feel less theoretical than it might otherwise sound.

What happens next?

The letter writers back international action on AI risk, the same broad conclusion Garton Ash reached. Where they push back is on urgency and framing. Waiting for a single dramatic warning shot may mean the warning never comes in a form anyone recognises in time.

For anyone thinking about this practically: the question to ask of any AI tool you use at work is not "has it caused a problem?" but "what safeguard did we remove to make this run smoothly?" That is the decision worth tracking.

Takeaway: Write down, in plain language, which human checks your workplace has removed since adopting any AI tool. If you cannot answer that question, someone should find out.