Editorial illustration for Google DeepMind's AI Co-Scientist Writes Plausible but Inaccurate Methods in Papers
DeepMind's AI Co-Scientist Generates Fake Research Methods
Google DeepMind's AI Co-Scientist Writes Plausible but Inaccurate Methods in Papers
Google DeepMind first showed off Co-Scientist in February 2025 as a hypothesis generator built on Gemini 2.0, with acknowledged gaps in fact-checking and literature review. Nine months later, the company says it has turned that tool into something closer to a working lab partner. The system now runs on current Gemini models and handles the full research loop itself: deriving hypotheses, drafting experimental plans, writing code or machine-readable lab protocols, analyzing what comes back, and producing manuscripts describing the results.
Google tested the upgraded system across three fields, each with a different level of human involvement. In materials science, Co-Scientist designed synthesis recipes for a semi-automated furnace that researchers had to run by hand. In biology, it built a prediction pipeline with expert feedback along the way.
In computer science, it worked without any human input at all. DeepMind also added verification modules meant to catch fabricated numbers by checking claims in generated text against the actual execution logs from the code.
The first test case, growing a hard-to-produce 2D material, shows both how far the system got and where it stalled.
Built on current Gemini models, Co-Scientist now plans experiments, writes code, and controls lab equipment instead of just generating hypotheses. What's technically new is the closed-loop research workflow: the system derives hypotheses from a research question, creates experimental plans, programs, or machine-readable lab protocols, analyzes results, and generates scientific manuscripts.
Why this matters
The gap Schmidgall flagged, methods sections that describe experiments the code never actually ran, is the kind of failure that doesn't show up until someone tries to reproduce the work. For researchers already drowning in papers with irreproducible results, an AI co-author that writes plausible fiction about its own process is a liability dressed up as a productivity gain. Google is right that letting Co-Scientist plan experiments and run equipment across three disciplines is a real step past hypothesis generation.
But "experimentally validated" and "trustworthy record of what happened" are different claims, and the second one is where science actually lives. If we're going to let these systems touch lab equipment and publish under their own byline, the verification tooling needs to mature at the same pace as the automation, not trail behind it. Until someone builds a reliable way to audit what the model did versus what it wrote, treat every Co-Scientist paper as a draft that needs a human checking the code against the claims, not a finished result.
Related Reading
Browse all Google news →
Research & Benchmarks Google's FACTS benchmark shows 70% factuality ceiling across four tests
Research & Benchmarks Databricks finds multi-step agents beat single-turn RAG by 21% to 38% on STaRK
Research & Benchmarks Nvidia's DLSS 4.5 beta adds 6x Multi Frame Generation for RTX 50 GPUs
Related Topic NVIDIA and Google Cloud let developers scale AI from prototype to production
Market Trends
Related Topic Google tests visual 'magazine-style' UI for Gemini 3 Pro users
LLMs & Generative AI
From Archives Cohere's Parse 5 Scores 79.2 on ParseBench, Winning on Cost
From Archives China Sees AI Agent Behavior as Key to Higher Value
From Archives Google AI Claims 2.6% Average WER for Gemini 3.5 Transcribe
Open Source
From Archives Google’s AI Notebook app adds full-text access to over 100,000 books
LLMs & Generative AI
Common Questions Answered
What capabilities does Google DeepMind's Co-Scientist now have compared to its February 2025 version?
The updated Co-Scientist has evolved from a simple hypothesis generator into a full research partner that can plan experiments, write code, control lab equipment, and generate scientific manuscripts. Built on current Gemini models, it now handles the complete closed-loop research workflow including deriving hypotheses, creating experimental plans, analyzing results, and writing lab protocols instead of just generating initial hypotheses.
What is the main problem with Co-Scientist's methods sections in scientific papers?
Co-Scientist writes plausible but inaccurate methods descriptions that describe experiments the code never actually ran, creating a significant reproducibility issue. This type of failure only becomes apparent when other researchers attempt to reproduce the work, making it a major liability for scientific integrity.
How does Co-Scientist's closed-loop research workflow function across different research disciplines?
Co-Scientist operates through an integrated workflow that derives hypotheses from research questions, creates experimental plans and machine-readable lab protocols, programs necessary code, analyzes the results returned, and generates scientific manuscripts. This system can handle research across three different disciplines while maintaining the ability to control lab equipment directly.
Why is the gap between Co-Scientist's written methods and actual executed experiments particularly problematic for the research community?
Researchers are already struggling with papers containing irreproducible results, and an AI co-author that generates convincing but fictional descriptions of its own experimental processes exacerbates this crisis. The issue is that these plausible inaccuracies mask fundamental failures in the research process and undermine scientific reproducibility.
Further Reading
- Accelerating scientific breakthroughs with an AI co-scientist - Google Research Blog
- Accelerating scientific discovery with Co-Scientist - arXiv
- When AI Co-Scientists Fail: SPOT-a Benchmark for Automated Verification of Scientific Research - arXiv
- New 'AI scientists' are improving—but reveal their fundamental limits - Phys.org
- Co-Scientist: A multi-agent AI partner to accelerate research - Google DeepMind
