One in three dangerous AI coding commands slip past human reviewers, game data shows
A browser game simulating real AI coding agent requests found that players approved roughly a third of malicious commands across more than 40,000 runs. The numbers point to a problem already showing up in production.
Key points
- Players approved approximately one in three malicious AI coding agent requests across more than 40,000 game runs and 409,000 total commands reviewed.
- The single most-missed dangerous command, "npm run analyze", was approved nearly 65 percent of the time despite a visible warning in the game's history log.
- Anthropic's own telemetry shows real users approve around 93 percent of permission prompts from Claude Code, its AI coding tool.
- Claude Code's "auto mode" catches roughly 83 percent of overeager or dangerous behaviours before they run, leaving about 17 percent to reach a human reviewer.
- Belgian developer Alex Wauters built the game in late May 2025 after noticing that developers either approve everything blindly or skip permission checks entirely.
The most honest stress test of AI coding safety right now is a browser game you can finish in under a minute.
The game, built by Belgian software developer Alex Wauters, puts you in the chair of a human reviewer watching an AI coding agent (software that carries out complex programming tasks step by step on your behalf) ask for permission to run commands. You have 60 seconds. Approve the safe ones. Block the dangerous ones.
Across more than 40,000 runs, players approved roughly one in three malicious requests. Not a great number.
Why do humans keep approving dangerous commands?
Fatigue is the short answer. Reviewing every action an AI agent takes is genuinely exhausting, and when you are drowning in prompts, attention slips.
Wauters built the game after noticing a pattern among developers using Claude Code. Faced with a flood of permission pop-ups, many simply turned on "dangerously-skip-permissions", which lets the model run without asking for human sign-off. Others clicked approve on everything without reading it. Neither is safe.
"The high amount of noise introduces fatigue, and developers don't always have the context of what has changed to quickly determine the risk," Wauters wrote in a blog post published alongside the game.
The data backs that up. Scope violations, things like an agent quietly reading AWS credential files or Kubernetes configuration data (files storing login keys and server settings, exactly what an attacker wants), were missed 35 percent of the time. Obviously catastrophic commands, like ordering deletion of an entire file system, got caught most often. People still recognise a bomb.
The subtler threats slipped through. Curl requests to unknown web addresses and typosquatted packages (software with names deliberately misspelt to resemble trusted ones, designed to trick people into running malicious code) were missed nearly as often as the credential grabs.
| Command type | Approximate miss rate |
|---|---|
| Scope violations (e.g. Reading credential files) | 35% |
| Typosquatted packages | ~35% |
| Curl requests to unknown APIs | ~35% |
| Crontab injections / git config hijacks | Lower |
| Obviously destructive commands (e.g. Delete root) | Lowest |
The single worst result: "npm run analyze", a command that sounds routine but can be configured to run any code hidden inside a project file. Nearly 65 percent of players approved it. The game displayed a warning in the history log directly above the approval button. Two thirds did not read it.
Is this actually happening outside the game?
Yes. Anthropic confirmed it with its own data.
In a May 2025 post on how Claude Code handles permissions, the company said users approve around 93 percent of all permission prompts the tool generates. "The more approvals a user sees, the less attention they pay to each," Anthropic wrote. Our coverage of Claude Code has flagged agent misbehaviour as a recurring theme since July 2026, and this data fits that pattern uncomfortably well.
Anthropic's answer is "auto mode", a system that uses a separate AI classifier (a model trained to judge whether a command looks risky) to filter dangerous requests before they reach a human. It catches around 83 percent of problematic actions. The remaining 17 percent still land on a human desk.
Anthropic is clear that auto mode is "one layer of defense-in-depth inside a sandbox, not a substitute for one."
Wauters agrees. His recommendation: run coding agents inside isolated environments called sandboxes or devcontainers (sealed-off digital workspaces that limit what an agent can touch on your real machine), use tools like auto mode, and write custom hooks (small scripts that intercept commands before they execute) to catch anything suspicious before it runs.
"We need to make the tooling easier to make these systems safer than pointing to human review as a valid solution," he told The Register AI.
Should you worry?
If you hand an AI agent a long task and walk away, what comes back might not be what you expected. The real story here is not that humans are bad reviewers under pressure. It is that the industry designed a workflow that assumes they will be good ones, and nobody checked that assumption until a browser game did it for them. The 83 percent catch rate from auto mode is genuinely useful, but a tool that still lets 17 percent of dangerous actions through cannot be the last word on safety. Watch what Anthropic and others build into the permission layer next.
