Editorial illustration for AI Researchers Grapple With Limits of Automating Empirical Science
AI Researchers Hit Wall Automating Science
AI Researchers Grapple With Limits of Automating Empirical Science
Eric and Wendy Schmidt's philanthropic outfit, Schmidt Sciences, brought its AI2050 fellows together in Mountain View, California, last week, packing a hotel 30 miles south of San Francisco with academics whose careers now hinge on a technology their own universities can barely afford to study. The program funds researchers working on AI, and its roster reads like a shortlist of the field's most cited names. Four years of large language model progress have shifted the center of gravity in AI research away from campuses and toward companies like OpenAI and Anthropic, which control both the compute and the model access that frontier work now requires.
University labs, unable to match corporate GPU budgets and locked out of proprietary systems, are stuck figuring out what empirical AI science even looks like when the tools of the trade sit behind a corporate firewall. MIT Technology Review's Will Douglas Heaven attended the gathering, hosting roundtable interviews and a media training session, and watched fellows wrestle openly with that mismatch. What emerged wasn't despair so much as a scramble to redefine what academic AI research can still contribute, and where it no longer competes on equal footing.
It’s a weird time for university AI researchers, who make up most of the AI2050 group. In the past four years, AI research has reoriented around large language models, and its cutting edge has moved from academic institutions to private companies.
Why this matters
The gap between "AI solved a math olympiad problem" and "AI ran a biology lab" is wider than most funding pitches admit. Math rewards clean, verifiable proofs; empirical science runs on messy data collection that no amount of compute shortens. Dettmers's optimism is worth noting precisely because he's a working scientist, not a lab pushing product. That's a useful check against the narrative that automation is coming for research wholesale.
For researchers and founders building AI-for-science tools, the AI2050 gathering is a reminder to separate the reasoning bottleneck from the data bottleneck. If your product promises to speed up discovery, ask which one it's actually attacking. Tools that generate hypotheses or crunch existing datasets faster are solving a different problem than tools that still need someone to run the experiment, wait for results, and repeat.
Conflating the two is how pilots get overpromised and underdelivered. Watch for whether Schmidt Sciences and similar programs start funding wet-lab automation specifically, since that's the actual constraint everyone's dancing around.
Related Reading
Research & Benchmarks Google's FACTS benchmark shows 70% factuality ceiling across four tests
Research & Benchmarks Databricks finds multi-step agents beat single-turn RAG by 21% to 38% on STaRK
Research & Benchmarks Nvidia's DLSS 4.5 beta adds 6x Multi Frame Generation for RTX 50 GPUs
From Archives FineBooks Aims to Fix Old OCR Text for AI Training at Scale
From Archives Hidden PDF Text Can Steal Data via Atlassian's AI Agent Rovo
Common Questions Answered
Why has the center of gravity in AI research shifted away from academic institutions?
Over the past four years, large language model progress has reoriented AI research around private companies rather than universities, which can barely afford to study these technologies. This shift has created a challenging environment for university AI researchers whose careers now depend on technologies their own institutions lack the resources to adequately investigate.
What is the difference between AI solving math problems and automating empirical science according to the article?
The gap between AI solving math olympiad problems and AI running a biology lab is significantly wider than most funding pitches acknowledge. While math rewards clean, verifiable proofs, empirical science relies on messy data collection that cannot be shortened by increased computational power alone.
What role does Schmidt Sciences' AI2050 program play in supporting academic AI research?
Schmidt Sciences' AI2050 program funds researchers working on AI and brings together academics whose careers now depend on AI technology, despite their universities having limited resources to study it. The program's roster includes some of the field's most cited researchers and represents an effort to support academic institutions grappling with the shift of cutting-edge AI research to private companies.
Why is a working scientist's perspective on AI automation more valuable than a lab's perspective?
A working scientist's optimism about AI automation serves as a useful check against the narrative that automation is coming for research wholesale, since they have direct experience with the practical challenges of empirical science. In contrast, labs pushing products may have incentives to overstate the capabilities of AI in automating research processes.
Further Reading
- Towards end-to-end automation of AI research - Nature
- How Far Are AI Scientists from Changing the World? - arXiv
- Position: AI Scientists Are Not Yet Ready for Open-Ended and Fully Autonomous Scientific Discovery - OpenReview
- Agentic AI Scientists Are Not Built For Autonomous Scientific Discovery - arXiv
- What are the limits to biomedical research acceleration through general-purpose AI? - Nature Scientific Reports
