AI test scores are hiding something. A new tool from Allen AI just exposed it
Researchers built software that looks inside AI benchmark tests, question by question, and found that some widely trusted scores are measuring very different things than advertised.
Key points
- Allen Institute for AI released BenchMIRT in 2025, a tool that audits the individual questions inside AI benchmark tests.
- BenchMIRT analysed results from 100 AI models across 16 benchmarks and more than 34,000 questions.
- The tool independently discovered that most AI tests cluster around two core skills: general reasoning and safety.
- BBQ, a popular test for social bias, turned out to measure reasoning ability more than safety behaviour.
- BenchMIRT predicted whether a model would answer an unseen question correctly 79% of the time, compared to 70% for simpler methods.
When a company tells you its AI model scored 85% on a safety test, that sounds clear. But what if half the questions in that test were actually measuring something else entirely? That is the problem a team at the Allen Institute for AI set out to fix.
Their tool is called BenchMIRT. A benchmark, in plain terms, is a standardised test researchers give to AI models to measure a specific skill, like whether the model refuses dangerous requests or handles tricky reasoning questions. BenchMIRT is software that looks inside those tests, question by question, and works out what each question is really measuring.
Why does this matter to anyone outside AI research?
AI benchmark scores shape real decisions. Companies use them to choose which AI tools to buy. Regulators use them to decide which models are safe to deploy. Doctors, teachers and employers are already using AI systems that were approved partly on the strength of these scores.
If the scores are blurry, the decisions built on them can be wrong.
BenchMIRT found several cases where popular tests were blurry. BBQ, which is supposed to measure whether an AI relies on social stereotypes, turned out to align much more closely with general reasoning ability in BenchMIRT's analysis. A model that scores poorly on BBQ may simply be bad at following the logic of a question, not necessarily biased.
WMDP, a test that checks whether a model knows dangerous chemistry, biology or cybersecurity information, showed a similar twist. Stronger reasoning was linked to lower WMDP scores, because the test rewards refusing to provide dangerous answers. A very capable model that correctly refuses scores well, but the skill doing the work is reasoning, not safety instinct.
What BenchMIRT actually does
The tool borrows from a field called psychometrics, the science of measuring human abilities through test responses. Psychometricians have long known that not every exam question tells you the same amount about a student. Some questions separate the top performers from the rest; others are noise.
BenchMIRT applies that logic to AI tests. It trains on how 100 different AI models answered more than 34,000 questions and then estimates, for each question, how hard it is and how well it separates strong models from weak ones. Crucially, the researchers never told it which tests were meant to measure safety and which measured reasoning. It worked that out on its own, and it landed on the same two categories every time it ran.
| What BenchMIRT found | Detail |
|---|---|
| Models and questions analysed | 100 models, 16 benchmarks, 34,000+ questions |
| Capabilities it discovered independently | Safety and general reasoning |
| Prediction accuracy on unseen questions | 79% correct |
| Simpler baseline accuracy | 70% correct |
| Questions needed to get a reliable signal | As few as 10% of a benchmark's full question set |
That last row is striking from a cost angle. Running an AI model through thousands of questions takes time and computing power. BenchMIRT suggests you can cut the question set by 90% and still get a reliable picture of how capable the model is.
What happens next?
The full technical paper is available via the Allen Institute, and the datasets are published on Hugging Face for other researchers to use.
For now, BenchMIRT is a research tool, not a product you can buy. But its findings put pressure on anyone who publishes or relies on AI benchmark scores to be more specific about what a score actually covers.
The honest takeaway: before you trust an AI system's test results, ask which questions made up that score, and what those questions were really testing. A single number rarely tells the whole story.
Common questions
Does this mean current AI safety tests are unreliable?
Not unreliable, but imprecise. BenchMIRT shows that some tests mix together different skills, which makes a single score harder to interpret. The tests are not broken; their scores just need more context.
Will this change how AI companies report their model performance?
Not immediately. BenchMIRT is a research method, not an industry standard yet. But if regulators and buyers start demanding question-level breakdowns, reporting norms could shift.
