AI agents are being trusted with more decisions than companies can actually verify

Key points

• A June 2026 survey of 157 enterprise teams found half had shipped an AI agent or feature that passed internal testing and then caused a customer-facing failure.
• One in four of those companies had experienced that kind of failure more than once.
• 66% of surveyed enterprises already allow some AI deployment without human review, or plan to within 12 months.
• Only 5% of respondents said they fully trust the automated testing systems that decide whether an AI is ready to ship.
• Larger companies, those with 2,500 or more employees, are moving fastest toward zero-human oversight and also report more customer failures than smaller firms.

Picture a quality inspector who admits, quietly, that the checklist is mostly wrong. That's roughly the situation a survey published by VentureBeat describes across a broad slice of corporate AI teams right now.

The June 2026 VB Pulse survey covered 157 people at companies with 100 or more employees, asking how they test and release AI agents: software programs that can carry out multi-step tasks on their own, handling a customer refund or sending a message without a human pressing send. It was self-selected rather than randomly sampled, so treat the numbers as a signal, not a census.

The signal isn't comfortable. Half the companies had already shipped an agent that cleared every internal test and still hurt a real customer. A quarter had done it more than once.

Why does a passing test not guarantee a working agent?

Traditional software testing checks whether a fixed input produces an expected output. Agents don't work that way. An agent chooses its own sequence of steps, calls external tools, pulls in live data, and may behave differently each run, even on the same task.

It can draft a valid refund and send it before approval arrives. It can clear five steps cleanly and leak sensitive information on the sixth. Each individual decision looks fine; the outcome isn't.

Respondents understand the gap. The most common reason for distrusting automated testing, cited by 29%, was that test scores don't align with what actually happens in production. Bias or inconsistency came next at 21%, lack of explainability at 18%, and privacy concerns at 17%.

We've tracked this theme closely: our 14 July story "Enterprises Deployed AI Agents Without the Safety Nets. Now They're Paying to Retrofit." found similar findings in a separate survey of 573 technical leaders, with two-thirds of firms already racing toward zero human oversight.

The US National Institute of Standards and Technology made a related point in its Generative AI Profile guidance: behaviour measured in a controlled environment often doesn't hold once real users and live data enter the picture. Field testing and ongoing monitoring after launch, not just a pre-release score, are what its guidance calls for.

Should you worry about companies moving fastest?

Yes, particularly the biggest ones. Companies with 2,500 or more employees are moving toward zero-human deployment at 70%, compared with 64% for smaller firms, and they're also reporting higher customer failure rates: 54% versus 48%. Speed and scale don't appear to be helping.

And yet autonomy keeps expanding. Two thirds of all surveyed companies are already running, or building toward, deployments with no human reviewer at all. Only 5% fully trust the automated checks that would replace that reviewer. Growing autonomy sitting on shaky assurance is the core problem here.

Anthropics guidance on agent evaluation draws a useful line: a system that succeeds once is not the same as one that succeeds reliably. For anything touching customers, money, or data, reliability is the only bar that matters.

Practically, that means running every scenario multiple times, varying the wording and context, simulating tool failures, and turning every real incident into a permanent test case. Zero-human oversight should be earned by demonstrated consistency, then bounded tightly by the consequences of failure. Drafting an internal summary isn't the same risk as approving a financial transaction, and companies that treat them the same are borrowing trouble.

Removing a human from a process doesn't remove the risk. It just removes the person who might have caught it.