One in three dangerous AI coding commands slip past human reviewers, game data shows
A browser game simulating real AI coding agent requests found that players approved roughly a third of malicious commands, even across more than 40,000 runs. The data points to a problem that goes well beyond a game.
Key points
- Players approved approximately one in three malicious AI coding agent requests across more than 40,000 game runs and 409,000 total commands reviewed.
- The single most-missed dangerous command, "npm run analyze", was approved nearly 65 percent of the time despite a visible warning in the game's history log.
- Anthropic's own telemetry shows real users approve around 93 percent of permission prompts from Claude Code, its AI coding tool.
- Anthropic's "auto mode" for Claude Code catches roughly 83 percent of overeager or dangerous behaviours before they run, but about 17 percent still get through.
- Belgian developer Alex Wauters built the game in late May 2025 after noticing that most developers either approve everything blindly or skip permissions checks entirely.
Here is a thing you probably did not expect to read today: the most honest stress test of AI coding safety right now is a browser game you can finish in under a minute.
The game, built by Belgian software developer Alex Wauters, puts you in the chair of a human reviewer watching an AI coding agent, a piece of software that carries out complex programming tasks step by step on your behalf, ask for permission to run commands. You have 60 seconds. Approve the safe ones. Block the dangerous ones. Score as high as you can.
Across more than 40,000 runs, players approved roughly one in three malicious requests. That is not a great number.
Why do humans keep approving dangerous commands?
Fatigue is the short answer. Reviewing every single action an AI agent takes is genuinely exhausting, and when you are drowning in prompts, your attention slips.
Wauters built the game after noticing a pattern among developers using Claude Code, Anthropic's AI coding tool. Faced with a flood of permission pop-ups, many simply turned on a setting called "dangerously-skip-permissions", which lets the model run without asking for human sign-off at all. The other camp clicked approve on everything without really reading it.
Neither approach is safe.
"The high amount of noise introduces fatigue, and developers don't always have the context of what has changed to quickly determine the risk," Wauters wrote in a blog post published alongside the game's late May launch.
The game data, first reported by The Register AI, backs that up in uncomfortable detail. Scope violations, things like an agent quietly reading AWS credential files or Kubernetes configuration data (files that store login keys and server settings, the kind of thing an attacker would love to steal), were missed 35 percent of the time. Obviously catastrophic commands, like ordering the deletion of an entire file system, got caught most often. People still know a bomb when they see one.
The trickier threats slipped through. Curl requests to unknown web addresses and typosquatted packages (software packages with names deliberately spelled almost like trusted ones, designed to trick people into running malicious code) were missed nearly as often as the credential grabs.
| Command type | Approximate miss rate |
|---|---|
| Scope violations (e.g. reading credential files) | 35% |
| Typosquatted packages | ~35% |
| Curl requests to unknown APIs | ~35% |
| Crontab injections / git config hijacks | Lower |
| Obviously destructive commands (e.g. delete root) | Lowest |
The single worst result: "npm run analyze", a command that sounds routine but can be configured to run any code whatsoever hidden inside a project file. Nearly 65 percent of players approved it. The game even displayed a warning in the history log directly above the approval button. Two thirds did not read it.
Is this actually happening outside the game?
Yes. Anthropic confirmed it with its own data.
In a May 2025 post about how Claude Code handles permissions, Anthropic said users approve around 93 percent of all permission prompts the tool generates. "The more approvals a user sees, the less attention they pay to each," the company wrote.
Anthropics answer is "auto mode", a system that uses a separate AI classifier, a model trained to judge whether a command looks risky, to filter out dangerous requests before they ever reach the human. It catches around 83 percent of problematic actions. The remaining 17 percent still land on a human desk.
Anthropric is clear that auto mode is "one layer of defense-in-depth inside a sandbox, not a substitute for one."
Wauters agrees. His recommendation is to run coding agents inside isolated virtual environments called sandboxes or devcontainers (sealed-off digital workspaces that limit what the agent can touch on your real machine), use tools like auto mode, and write custom hooks (small scripts that intercept commands before they execute) to flag anything suspicious before it runs automatically.
"We need to make the tooling easier to make these systems safer than pointing to human review as a valid solution," he told The Register AI.
If you hand an AI agent a long task and walk away, what comes back might not be what you expected. Checking in matters, and understanding what you are checking matters more.
