AI Novelty Judges Are Too Unreliable to Trust, New Benchmark Shows

Key points

• Changing a few words in a prompt caused GPT-5.4's accuracy at identifying novel research ideas to collapse from 93.5% to 40.9%, below random chance, according to a study published on arxiv.org in October 2026.
• Ai2 tested six models across two evaluation settings and found no prompt that worked reliably across all of them.
• The benchmark was built from ICLR 2026 peer reviews, selecting only papers where every reviewer agreed on whether the work was original.
• Two purpose-built novelty evaluation systems performed worse than a simple prompted Claude Opus 4.6 baseline.

Scientists are increasingly asking AI to generate new research ideas. The harder question is how to check whether those ideas are actually new.

The standard answer is to ask another AI. These so-called novelty judges, large language models (the technology behind chatbots like ChatGPT and Claude) prompted to rate whether a research idea is original, have become a routine part of AI-assisted discovery pipelines. Researchers at the Allen Institute for AI (Ai2) decided to put the judges themselves on trial. What they found should give pause to anyone using these tools.

How bad is the problem?

Pretty bad. Ai2 tested six models: GPT-5.1, GPT-5.2, GPT-5.4, Claude Sonnet 4.5, Claude Opus 4.5, and Claude Opus 4.6. In the starkest result, the team showed GPT-5.4 identical pairs of ideas under two slightly different prompts. One told the model that reviewers had confirmed exactly one idea in each pair was novel; the other simply asked which idea was more novel. That single change sent accuracy from 93.5% down to 40.9%, and the model reversed its verdict on more than half the pairs.

This isn't a one-model quirk. Across all six judges, small prompt tweaks produced large swings, retrieval tools (giving the judge access to related papers) didn't consistently help, and additional reasoning time also failed to stabilise results. No single prompt worked well across every model tested.

For ordinary readers this matters because automated science tools that promise to speed up drug discovery or materials research often rely on exactly this kind of AI judge to filter out unoriginal ideas before a human ever sees them. If the filter is unreliable, bad ideas get through and good ones get dropped, silently.

Where did the benchmark data come from?

Ai2 built the test set from ICLR 2026 peer reviews, the formal written assessments that researchers submit when judging whether a paper deserves to be published at a major AI conference. The team kept only papers where every reviewer agreed on originality: 154 accepted papers whose reviewers unanimously praised the novelty, and 145 rejected papers whose reviewers unanimously questioned it. A human annotator spot-checked the extraction process and agreed with the automated labels in 47 of 50 cases.

That is a more careful ground truth than most previous work, which validated novelty judges on human-written papers rather than on the AI-generated ideas those judges are actually deployed to assess. Judges behaved more erratically on AI-generated ideas than on human-authored ones, which means a judge that looks fine on a standard test could still fail badly in real use.

Ai2 released the benchmark alongside the paper, so other teams can run their own tests. This is the kind of infrastructure work the field has been missing.

This finding lands at an interesting moment for Ai2's broader ambitions. We reported on 23 October that LitPivot, an Ai2 tool, rewrites your research idea as you read the papers behind it, and on 2 October that Ai2 and Hugging Face released AstaBrief 8B, a small model for generating cited scientific reports. Both tools sit upstream of exactly the novelty-evaluation step this paper says nobody has properly stress-tested.

The plain truth: the AI science community has been building elaborate discovery pipelines on top of evaluation tools that haven't been rigorously tested. This paper is the first serious attempt to measure the problem. The numbers are a reason to slow down before trusting any automated novelty score at face value.

Common questions

Does this mean AI cannot be used to evaluate research at all?

Not quite. The study shows current novelty judges are unreliable enough that their verdicts shouldn't be trusted without human review. It doesn't show that a reliable judge is impossible, just that nobody has built one yet.

Which model performed best overall?

No single model came out on top across all conditions. A simple prompted Claude Opus 4.6 outperformed two purpose-built novelty evaluation systems, but even it showed large swings when prompts changed, making it hard to call any one model the safe choice.