#benchmarks
6 stories taggedbenchmarks.
Speech AI Gets Its First Major Hindi Benchmark, and It Measures Who Gets Left Behind
A new evaluation dataset called Monsoon puts Hindi and Indian English on the global leaderboard for the first time, with nearly 5,000 speakers and data designed to expose the gaps that single-score tests hide.
Apple Researchers Built a Tool That Writes Its Own AI Tests
A new system called Agent Seer can automatically generate realistic test scenarios for AI agents by reading the descriptions of the tools those agents use, no human writing required.
Google DeepMind puts its AI in a cryptographic sealed box for testing
A first-of-its-kind double-blind evaluation aims to stop AI models from secretly studying the exam before test day.
Five million people rank AI designs so machines can learn what looks good
A startup called Intelligence has raised $7.9 million to run a platform where ordinary users vote on AI-generated images and websites, and the data it collects is already worth $60 million a year to the companies that build AI models.
Researchers Build a Test to See If AI Can Actually Summarise a 16-Minute Video
A new benchmark called LVSum reveals how badly today's AI video tools lose track of when things happen in long recordings.
An AI Taught Itself to Write Better Code, No Teacher Required
A surprisingly simple trick let a leading AI model lift its coding score by nearly 13 points, just by studying its own answers.