#AI benchmarks
26 stories taggedAI benchmarks.
AI Translation Judges Struggle With the Same Errors Human Reviewers Do
A new study from Apple ML Research finds that large language models are only a partial replacement for human graders when checking the quality of machine-translated text, with both groups stumbling on the same hard cases.
Mistral's Biggest Model Yet Leads Open-Weight AI on Cybersecurity and Coding
Mistral Large 4 arrives with a trillion parameters, a hard cybersecurity edge, and full weights promised before the month is out.
Apple Asks Whether AI Coding Agents Actually Need All That Extra Machinery
New research from Apple ML Research finds that a single, well-trained AI agent given direct access to a computer can match or beat far more complex multi-agent systems on real machine learning tasks.
Nvidia's Kumo Tabular Reads a Spreadsheet Cold and Beats Every Tuned Model Doing It
A new open AI model from Nvidia makes predictions on business data tables without any training on your data, no setup, and no specialist required, and it just topped every major leaderboard.
Paying for a frontier AI model now buys you about four months of advantage
A new Mozilla report puts hard numbers on the gap between the priciest closed AI and the best free-to-download models. For most everyday work, the gap is shrinking fast.
A mystery Chinese AI model spent a week fooling the internet. Now it wants 45% of your AI budget.
GLM-5.3-Flash launched anonymously, served trillions of tokens for free on Chinese chips, and costs a fraction of US rivals. That combination is forcing companies to rethink what they actually need to pay for.
AI aces trivia but fumbles riddles: the puzzle tests that expose what machines still can't do
From mental rotation to river-crossing logic, a new set of benchmarks shows where AI trips over problems any sharp human can crack, and where it beats us cold.
A tiny AI agent from a London startup just beat Anthropic and OpenAI at reading scientific papers
Inherent, founded by four Google DeepMind veterans, says its Faraday agent outperformed much larger models from Anthropic and OpenAI on a key science benchmark, despite running on a model roughly a tenth of their size.
GLM-5.3 Opens Up to Developers at the Same Price as Its Predecessor
Chinese startup Z.ai's new frontier model is now available via API, matching the old per-token rate while scoring higher on independent benchmarks. But a quirk in how the model writes means your bill may still rise.
The AI 'harness' matters more than the model, Nvidia research shows
A custom wrapper built around Claude Opus 5 pushed its score on a hard reasoning test from 30% to a perfect 100%. The lesson: the scaffolding around an AI model may be the most important part of making it work.
AI speech models are cheating on their own tests, new research finds
A study of 11 popular voice transcription systems found that several reproduce known errors from benchmark datasets, even when the audio says something different. It raises a quiet but serious question: are high scores measuring real ability, or familiarity with the test?
A 27-billion-parameter AI model you can run at home just matched cloud-only rivals on coding tests
Alibaba's Qwen3.8-27B is free to download, fits on a high-end laptop, and scored level with mid-tier OpenAI and Anthropic models on independent benchmarks. Developers are calling it the clearest sign yet that frontier-grade AI is moving off the cloud and onto personal hardware.
More Memory Does Not Always Mean a Smarter AI Agent
A study across eight AI models found that feeding an agent more of its own past experience can hurt as much as help. The right amount of memory depends entirely on how capable the model already is.
Google launches Gemini 3.7 Flash just three weeks after its predecessor
The new model brings measurable gains in coding and document reading, and arrives with a cut-price introductory rate as Google feels pressure from cheaper rivals.
Microsoft's Orchard framework lets small AI agents punch well above their weight
A new open-source toolkit from Microsoft Research trains AI agents that can fix code, browse the web, and manage tasks, using models far smaller than today's frontier giants.