UK Government AI Lab Publishes Benchmark Results Anyone Can Check
The UK AI Security Institute is releasing verified test results for six frontier AI models under a shared open standard, so researchers worldwide can see exactly how scores were produced and repeat the work themselves.
Key points
- The UK AI Security Institute released verified benchmark results for six frontier large language models, including Claude Opus 4.6 and GPT-5.4, covering five major capability tests and two cybersecurity evaluations.
- A new AISI paper, "How Inference Compute Shapes Frontier LLM Evaluation," finds that the amount of computing power a model uses during a test changes its apparent score, casting doubt on many headline benchmark comparisons.
- Results are published through Evaluation Cards, a shared platform developed with the EvalEval Coalition, giving every result a standard record of the setup conditions that produced it.
- AISI's OptStop framework reduces the number of test runs needed to reach a firm conclusion, according to a paper published on arxiv.org in August 2026.
When an AI company publishes a benchmark score, a large language model's (LLM's) performance number on a standardised test, readers usually have no way to know exactly how the test was run. The setup details matter enormously, and they've mostly stayed hidden.
The UK AI Security Institute (AISI), a research body inside the UK government's Department for Science and Innovation Technology whose mission is to equip governments with a scientific understanding of AI risks, is trying to fix that. Working with the EvalEval Coalition, a research community focused on evaluation science, AISI is publishing its test results through a shared platform called Evaluation Cards, first reported by Hugging Face.
What exactly did AISI release?
The release covers six frontier models tested across five well-known benchmarks. Think of a benchmark as an exam for AI: standardised questions, marked answers, a score at the end.
| Model | Benchmarks covered |
|---|---|
| Claude Opus 4 | HealthBench, FrontierMath, Humanity's Last Exam, SWE-Bench Pro, Terminal-Bench 2.0 |
| Claude Opus 4.5 | same five |
| Claude Opus 4.6 | same five |
| GPT-5 | same five |
| GPT-5.2 | same five |
| GPT-5.4 | same five |
Two cybersecurity evaluations, Cyber CTFs and The Last Ones, cover a partially overlapping set of models. Every result comes with the configuration details needed to reproduce it.
Why do the setup details change the score?
This is the heart of it. AISI's accompanying paper shows that inference-time compute, the amount of processing power a model uses while generating each answer, shifts scores on tests like Humanity's Last Exam. Give the model more compute per question and it does better. Two labs can run the same model on the same exam and get different numbers simply by choosing different settings. Comparing those scores as though they were equivalent is like comparing two runners' times when one ran downhill.
AI2Day has covered the broader pressure on AI infrastructure closely, including how Anthropic and OpenAI have been expanding their data centre footprint and how new chips are being purpose-built to handle it, from Nvidia's Vera Rubin to Qualcomm's AWS deal. What AISI's work adds is a clear argument that more compute doesn't just make models faster: it changes how they score on tests that policymakers use to make decisions.
The practical upside is AISI's OptStop framework. Rather than running every test item the same number of times, it keeps adding test runs only where results are still uncertain and stops early everywhere else. For safety researchers with limited budgets, that matters. The August 2026 arxiv paper details the methodology.
Our 17 September story "Your AI Agent Scored 77%. It Only Nails the Same Task Every Time on 53% of Them" put hard numbers on a related gap: the distance between a model that usually works and one that reliably works. AISI's release gives that problem a reproducibility layer.
Should ordinary people care about benchmark scores?
Yes, practically. Governments use benchmark results to decide which AI systems are safe to deploy in hospitals and courts. If those benchmarks are unverifiable, the safety decisions built on them are shakier than they look. Standardised, reproducible results give regulators a firmer footing.
The main thing to watch now is whether other evaluation organisations adopt the Every Eval Ever schema and publish their own Evaluation Cards. If they do, side-by-side comparisons across labs become possible for the first time. If not, AISI's release is a useful reference point but a lonely one.
Common questions
Does this mean current AI safety tests are unreliable?
Not exactly unreliable, but they're harder to compare than they look. Different setups produce different scores for the same model, and AISI's work makes that gap measurable for the first time.
Can the public access these results?
Yes. The Evaluation Cards platform is open, and AISI's paper accompanies the data so anyone can read both the scores and the conditions that produced them.
Is OptStop the same as making AI models cheaper to run?
Not directly. OptStop reduces the number of test runs needed to evaluate a model accurately. It cuts the cost of measuring AI performance, not the cost of running AI products day to day.
