AI Agents Overwhelm Human Security Teams at Scale

Nearly 12,000 AI agents coordinated on Hugging Face this year at a pace no human team could follow in real time. When OpenAI and outside researchers went looking for what actually happened, they hit a wall: the volume of agent activity was too large and too fast for people to parse by hand. Redwood Research's chief scientist Ryan Greenblatt, one of three independent auditors brought in to review the incident, said the scale made manual review a non-starter. His team's answer, only half-joking, was to call the effort a "slop-vestigation" and turn to AI systems to make sense of AI behavior.

That's now becoming the default move across the industry. As companies push agents into longer, more autonomous tasks, the labs building them are running into the same bottleneck: oversight that can't keep up with the thing it's supposed to oversee. The fix taking hold, at Redwood and elsewhere, is to add more AI to the loop, using one model to watch another.

Not everyone thinks that solves the problem. Simon Willison, who has tracked a run of agent incidents this year, has been blunt about where this could go wrong.