AI's biggest dangers won't announce themselves

Key points

• Experts writing to The Guardian argue that framing AI risk as an "AI Hiroshima" moment misrepresents how serious harm is likely to unfold.
• The gravest dangers may include AI helping design pathogens or find weaknesses in critical infrastructure, with humans still formally in charge throughout.
• Harm could also build gradually through thousands of small decisions: a little more automation here, one safety check removed there.
• By the time the danger becomes obvious, the letter writers warn, the decisions that mattered most may already have been made.

The word "Hiroshima" conjures a single, unmissable catastrophe. That is exactly the problem, say the letter writers responding to a Timothy Garton Ash column published in The Guardian.

Hiroshima was not a machine going rogue. People designed the bomb, people ordered its use, and people dropped it. The technology performed more or less as intended. Framing AI risk the same way, the letters argue, sets us up to miss the real threat.

So what does real AI danger actually look like?

It probably looks quieter than a mushroom cloud. Dr Simon Nieder, one of the correspondents, sketches two different paths to serious harm.

The first is specific and alarming. An AI system, meaning software trained on vast amounts of data to complete complex tasks, could help a bad actor design a dangerous pathogen (a disease-causing organism) or locate a weakness in power grids, water systems or financial networks. A human would still make the final call. No sci-fi robot uprising required.

The second path is slower and harder to see. Imagine a large organisation, a hospital, a government department, a bank, that gradually hands more consequential decisions to AI tools because the tools keep working well. Each handover looks sensible in isolation. One safeguard feels redundant, so it gets switched off. A task that used to need a senior sign-off now runs automatically because the system has a good track record.

None of those individual moments feels like a crisis. Together, they can add up to one.

Why does this matter for ordinary people?

Because the standard reassurance, "don't worry, a human is always in the loop", becomes less meaningful the more we trust the machine without checking it. If your bank, your hospital or your local council is quietly removing oversight steps because an AI system looks reliable, you may never know until something goes wrong.

This is also a survivorship-bias problem worth naming plainly. We hear about AI tools that worked. We rarely hear about the safety check that was quietly dropped three years ago and hasn't caused a problem yet.

What happens next?

The letter writers back international action on AI risk, the same broad conclusion Garton Ash reached in his original column. Where they push back is on urgency and framing. Waiting for a single dramatic warning shot may mean the warning never comes in a form anyone recognises in time.

For anyone thinking about this practically: the question to ask of any AI tool you use at work is not "has it caused a problem?" but "what safeguard did we remove to make this run smoothly?" That is the decision worth tracking.

Takeaway: The most useful thing you can do right now is write down, in plain language, which human checks your workplace has removed since adopting any AI tool. If you cannot answer that question, someone should find out.