AI Self-Improvement Loop: Lab Researchers Sound Alarm

Severin Field spent late summer 2025 interviewing 25 researchers at OpenAI, Anthropic, Google DeepMind, Meta, and several US universities about a narrow but consequential question: can an AI system build a better version of itself, then keep doing that in a loop? Twenty of the 25 people he talked to rated this kind of automated AI research among the most severe and urgent risks the field faces. Field wrote up the results for his newsletter, The Attack Surface, and the timing matters. Some of the milestones his interviewees named as markers of recursive self-improvement have already been reached in the months since he conducted the interviews.

The researchers Field spoke with kept returning to one yardstick: METR's Task Horizon benchmark, which tracks how long a task an AI agent can complete on its own before it needs help. That number has been doubling roughly every six months since 2019, and by some estimates the doubling time has shrunk to four months since 2024. Nobody Field interviewed disputed that AI systems are already improving AI systems in some sense. The argument was over whether those gains compound on their own, without a human in the loop, and what would have to break first for that to happen.