AI Spots Bugs in Minutes. Proving They Are Real Still Takes a Human.

Key points

• AI-assisted security tools can scan an entire codebase in minutes, a task that once took a skilled analyst a week.
• Bug bounty platforms, which pay cash rewards to researchers who find real software flaws, are reporting a surge in AI-generated reports that turn out to be fabrications.
• The curl open-source project maintainer has publicly described being overwhelmed by polished-looking bug reports that do not hold up under testing.
• A valid security finding requires a working demonstration, not just a written description, and that has always been true.
• Small business owners who receive unsolicited "critical flaw" emails should ask for a live demonstration on a test account before taking any action.

As first reported by ThreatVectr, the security research world is bumping hard against a simple problem: AI tools are very good at looking confident, and confidence is not the same as correct.

Here is how the new workflow actually looks. A security researcher opens an AI-assisted tool, which uses a large language model (the same type of technology that powers chatbots like ChatGPT) to read through software code. The tool summarises what the program does, flags functions that look suspicious, and even drafts sample attacks, called payloads, that the researcher can fire at a live system. What used to take days now takes an afternoon.

That speed is genuinely useful. Nobody is arguing otherwise.

The trouble is what happens next. A large language model does not know when it is guessing. It will describe a vulnerability in confident, technical language, invent a function name that does not exist in the actual code, and propose an attack that has never been tested on a real system. Every one of those reports then lands in someone's inbox.

What does this cost real people?

It costs time, which is the one thing security teams have the least of. Each bogus report has to be triaged, meaning a human analyst must sit down, read it carefully, try to reproduce the described flaw, and then write back explaining why it does not exist. Multiply that by hundreds of AI-generated submissions each week and you have burned through the very resource you were trying to protect.

Volunteers who maintain popular open-source software, tools that millions of people and businesses use every day, are feeling this hardest. The maintainer of curl, a widely used data-transfer tool, has said publicly that AI-generated hallucinations (fabrications a model produces with total confidence) are eating into the time available for real work.

Bug bounty platforms are reporting the same pattern. Pay researchers for real findings and suddenly there is a financial reason to flood the queue with AI-drafted guesses.

The standard that actually matters has not changed. A valid finding means a working demonstration: exact steps, exact software version, and ideally a proof-of-concept, a short piece of code or a specific input sequence that reliably triggers the flaw. Prose without proof is just a guess in a lab coat.

The one honest takeaway: if someone emails your business claiming they found a critical flaw in your website, ask them to demonstrate it on a test environment. Legitimate researchers expect that request. People fishing for a quick payout rarely follow through.