Editorial illustration for Google study: AI benchmarks ignore human disagreement; under 10 raters fail
AI Benchmarks Flawed: Google Reveals Rating Bias
Google study: AI benchmarks ignore human disagreement; under 10 raters fail
Most AI rankings are statistically worthless. They rely on a handful of human raters, which a new Google study shows is a fundamentally flawed way to measure anything meant for actual people.
Imagine a benchmark declaring one model better than another based on the votes of three people. That’s essentially the current standard. Google researchers tested thousands of budget and rater combinations and found fewer than ten raters per test example fails to produce reliable, reproducible comparisons.
The typical one to five raters used today ignores how people genuinely disagree, building rankings on sand. Reliable results need around a thousand total annotations, but only if that budget is split correctly between the number of test examples and the number of raters judging each one.
Now imagine asking 20 diners to rate the same 50 dishes. You'd walk away with a far richer picture of what's actually good and what isn't. Today's AI benchmarks overwhelmingly follow the first model, casting a wide net across test examples while collecting only a thin layer of human judgment for each one.
The industry has been grading on a non-existent curve. Benchmarks using a few raters pretend to measure objective quality but only capture a narrow slice of judgment. This erases the disagreement that tells us if a model works for anyone beyond a small, homogenous group.
The fix requires spending annotation money to reflect the messy reality of human preference. Otherwise you’re just building a mirage.
Related Reading
Browse all Google news →
Research & Benchmarks Google's FACTS benchmark shows 70% factuality ceiling across four tests
Research & Benchmarks Databricks finds multi-step agents beat single-turn RAG by 21% to 38% on STaRK
Research & Benchmarks Nvidia's DLSS 4.5 beta adds 6x Multi Frame Generation for RTX 50 GPUs
Related Topic NVIDIA and Google Cloud let developers scale AI from prototype to production
Market Trends
Related Topic Google tests visual 'magazine-style' UI for Gemini 3 Pro users
LLMs & Generative AI
From Archives Alibaba's Qwen team adds method that lengthens AI answers, prompting reasoning
From Archives Open models cross threshold; frontier models show per‑category correctness
From Archives Google Vids adds Veo, Lyria AI models and directable avatars for flyers, reels
LLMs & Generative AI
From Archives Google releases Gemma 4 under Apache 2.0, noting lower memory, near‑zero latency
Open Source
Common Questions Answered
How many raters does Google's study suggest are needed for reliable AI benchmark evaluations?
Google's research indicates that more than ten raters per test example are necessary for statistically reliable results. The study challenges current practices of using only one to five raters, demonstrating that such limited human input fails to capture the full range of human opinion and judgment.
What did Google's research reveal about the current methods of AI model comparisons?
The study found that typical AI benchmark evaluations using just three to five raters per example are insufficient for making reproducible model comparisons. By testing thousands of combinations across different budget sizes and rater counts, researchers discovered that around 1,000 annotations can yield stable results when carefully balanced.
Why are more human raters important in AI benchmark testing?
More human raters help capture the nuanced and diverse range of human opinions and judgments when evaluating AI models. The Google study emphasizes that fewer than ten raters can introduce significant noise and unreliability into benchmark comparisons, potentially leading to misleading conclusions about AI model performance.
