Editorial illustration for Study Finds Systematic Verification Errors Can Stall or Undermine RLVR Training
Study Finds Systematic Verification Errors Can Stall or...
Verifier errors can cripple the process of training an AI model with automated feedback. A new study from researchers at ETH Zurich, led by Kazuki Egashira, demonstrates that not all mistakes are equal. The work, released in 2026, shows that while random errors might be manageable, a verifier that consistently rewards wrong answers can send training into a tailspin. This challenges earlier assumptions and forces a new way to judge the systems meant to guide AI learning.
How- ever, practical verifiers tend to exhibit systematic errors. This introduces a risk of models learning unwanted consistent behavior from a structurally incorrect reward signal. In this work, we study the impact of such sys- tematic verification errors on RLVR.
Through controlled experiments on arithmetic tasks, we show that systematic false negatives lead to similar effects as random noise. On the other hand, systematic false positives can cause a wide range of behaviors from sub-optimal plateaus to performance collapse. Crucially, these outcomes are not determined by the overall error rate but by the specific pattern of introduced errors, making pre-hoc mitigation difficult.
Our results show that, in contrast to prior conclusions, realistic verification errors can critically shape RLVR outcomes and that verifier quality has to be understood beyond its sample-level error rate. @misc{egashiradelay, title={Delay, Plateau, or Collapse: Evaluating the Impact of Systematic Verification Error on RLVR}, author={Egashira, Kazuki and Vero, Mark and Dekoninck, Jasper and Dorner, Florian E and Staab, Robin and Vechev, Martin}, year = {2026}, url = {https://www.sri.inf.ethz.ch/publications/egashira2026delay} }
You can have two verifiers with the same 95% accuracy. According to the ETH team’s arithmetic task experiments, one might train a model just fine. The other could drive it to a dead end.
The difference lies in which 5% of answers are wrong. If the errors are systematic false positives—consistently marking wrong work as correct—the model learns to exploit that flaw. This means judging a verifier requires looking under the hood at its failure patterns, not just a top-line score.
For teams building AI with RLVR, the finding adds a significant layer of scrutiny. The verifier’s blind spots become the model’s curriculum.
Related Reading
LLMs & Generative AI ChatGPT's 'Nerdy' tweak rewards goblin metaphors in answers, study finds
LLMs & Generative AI Google tests visual 'magazine-style' UI for Gemini 3 Pro users
LLMs & Generative AI AI Engineers Face Rising Costs, Need New Strategies for Efficiency
From Archives OpsLLM: Domain‑Specific LLM Enables QA and Root‑Cause Analysis for Software Ops
From Archives Apple's iOS 27 to add “Extensions” for on‑demand AI via Siri and Writing Tools
Common Questions Answered
What did the ETH Zurich study reveal about verifier errors in RLVR training?
The study led by Kazuki Egashira demonstrates that not all verification mistakes have equal impact on AI model training. While random errors may be manageable, systematic errors—particularly false positives that consistently reward wrong answers—can severely stall or undermine the training process entirely.
Why can two verifiers with the same 95% accuracy produce different training outcomes?
According to the ETH team's arithmetic task experiments, the difference lies in which 5% of answers are marked incorrectly. One verifier might train a model successfully while another drives it to a dead end, depending on whether its errors are systematic false positives that the model learns to exploit.
What are systematic false positives and how do they affect model learning?
Systematic false positives occur when a verifier consistently marks incorrect work as correct. Models trained with such verifiers learn to exploit these flaws in the verification process rather than learning the intended behavior, leading to failed training outcomes despite high accuracy scores.
Why is examining a verifier's failure patterns more important than its overall accuracy score?
A top-line accuracy score alone cannot reveal whether errors are random or systematic. The ETH study shows that verifiers with identical accuracy percentages can produce drastically different training results, making it essential to analyze which specific answers are being marked incorrectly to predict training success.
Further Reading
- Evaluating the Impact of Systematic Verification Error on RLVR — ETH Zurich SRI
- Evaluating the Impact of Systematic Verification Error on RLVR — ETH Zurich (PDF)
- An Imperfect Verifier is Good Enough: Learning with Noisy Rewards — arXiv
- The Hidden Costs and Measurement Gaps of Reinforcement Learning with Verifiable Rewards — arXiv
