Editorial illustration for AI access drops correct answers by two-thirds, study shows
AI Access Cuts Correct Answers by Two-Thirds
People will admit they don't know something, until an AI chatbot offers to weigh in. That's the finding from a new study covering five experiments and 3,132 participants, which set out to test whether merely having access to a language model changes how willing people are to say "I don't know."
The researchers rigged the test deliberately. They asked questions about fine visual details from movies, like the color of a team uniform in Bend It Like Beckham, the kind of detail that almost never shows up in text the models were trained on. The AI system used, Step 3.5 Flash, got these nearly all wrong.
Three other systems, GPT-5.5, Claude 4.6 Sonnet, and Gemini 3.5 Flash, handled most other questions fine but stumbled on the harder ones too. That setup matters: since the AI advice was mostly wrong, any shift in behavior can't be chalked up to people sensibly trusting a reliable source.
What the researchers found instead was a near-total collapse in people's willingness to withhold judgment once AI advice entered the picture, regardless of whether that advice held up.
Researchers ran five experiments with 3,132 participants to test whether access to a language model changes how people handle uncertainty. Just having AI advice available nearly wiped out people's willingness to say "I don't know."
Why this matters
For anyone building products on top of language models, this study is a warning about what happens the moment users trust the interface more than their own judgment. Step 3.5 Flash was wrong almost every time on these movie-detail questions, yet its mere presence pushed participants from 27.5 percent accuracy down to 9.2 percent. That's not a model failing loudly.
It's a model failing quietly enough that people stop checking themselves. The financial incentives in studies 2 through 4 helped, but didn't come close to closing the gap, which tells us this isn't a motivation problem we can fix with better UX nudges or a "please verify" disclaimer. If money on the line doesn't restore people's willingness to say "I don't know," we should assume most production settings, where the stakes are softer and the interface is slicker, will be worse.
For researchers and founders shipping AI assistants into any domain with real consequences, medical, legal, financial, the lesson is blunt: availability itself changes behavior, independent of accuracy. Measuring model correctness in isolation misses the more important number, which is what happens to human judgment once the model is just sitting there, one click away.
Related Reading
Research & Benchmarks Microsoft AI Chief: "If It Isn't Safe We Shouldn't Build It
Research & Benchmarks Google's FACTS benchmark shows 70% factuality ceiling across four tests
Research & Benchmarks Nvidia's DLSS 4.5 beta adds 6x Multi Frame Generation for RTX 50 GPUs
From Archives Nvidia’s SoL-Pi halves coding agent token use with evidence-preserving reducer
From Archives OpenAI Notifies Dozens After Agents Posted 53 User Images
Common Questions Answered
What was the main finding of the study about AI access and people's willingness to say 'I don't know'?
The study of 3,132 participants across five experiments found that merely having access to a language model dramatically reduced people's willingness to admit uncertainty. Participants' accuracy on movie detail questions dropped from 27.5 percent to just 9.2 percent when an AI chatbot was available, even though the AI was providing incorrect answers almost every time.
How did researchers design the experiments to test AI's impact on admitting uncertainty?
Researchers deliberately rigged the test by asking participants questions about fine visual details from movies, such as the color of a team uniform in Bend It Like Beckham. These were the types of details that people almost never get right, allowing researchers to measure whether AI availability changed their confidence in saying 'I don't know.'
Why is this study important for product builders using language models?
The study serves as a warning that users may trust an AI interface more than their own judgment, even when the AI is providing incorrect information. When language models fail quietly rather than loudly, people stop checking themselves and their answers, leading to significantly reduced accuracy and decision-making quality.
What specific AI model was used in the experiments and how did it perform?
Step 3.5 Flash was the language model used in the study, and it was wrong almost every time on the movie-detail questions. Despite its poor performance, its mere presence was enough to push participants' accuracy down from 27.5 percent to 9.2 percent, demonstrating the powerful influence of AI availability on human confidence.
Further Reading
- Papers with Code Benchmarks - Papers with Code
- Chatbot Arena Leaderboard - LMSYS
