Editorial illustration for Security Researcher Finds AI Systems Repackage Existing Work as Original
Security Researcher Finds AI Systems Repackage Existing...
James Kettle has spent years poking at the guts of web servers looking for flaws nobody else has found. At Black Hat Las Vegas on Wednesday, he presented research asking a narrower question than most of the AI security talk this year: can an AI system dream up a genuinely new hacking technique on its own, start to finish, rather than just applying known tricks faster?
His answer split the difference. Left alone, the models Kettle tested showed little ability to invent an original attack path. Paired with a human steering at the right moments, though, they turned into something closer to a research partner, capable of surfacing ideas Kettle hadn't considered himself.
That partnership produced a concrete result. Kettle says the process led him to identify a previously undocumented category of vulnerability he's calling Shared-Parser Confusion, tied to how some web servers reuse the same code to handle both incoming requests and outgoing responses. He described the finding to WIRED before his talk, framing it as evidence of a real attack surface most defenders haven't been watching.
Importantly, though, when paired with human guidance and insight in key moments, Kettle found that AI is an extremely powerful partner in conceptualizing and uncovering new strategies for hacking.
Why this matters
Kettle's findings land at an awkward moment for anyone selling agentic AI as a research partner rather than a research assistant. Vendors keep pointing to real-world cases of AI-assisted hacking as proof these systems can think like attackers. What Kettle found is closer to a well-read intern with no citation discipline: capable of producing plausible-sounding, esoteric-seeming output that turns out to be existing work with the serial numbers filed off.
For developers and founders building on these tools, the practical lesson is blunt. If you're using agentic AI to generate novel exploit concepts or theoretical security work, you need a human who already knows the literature well enough to catch repackaging, because the system itself won't flag it. For researchers, this is a useful benchmark problem: originality claims from AI output need the same scrutiny as an unverified bug bounty submission.
The bigger story here isn't that AI can't hack creatively yet. It's that "creatively" is doing a lot of unearned work in how these capabilities get described, and Kettle's method for testing that claim is worth replicating elsewhere.
Related Reading
Research & Benchmarks Google's FACTS benchmark shows 70% factuality ceiling across four tests
Research & Benchmarks Databricks finds multi-step agents beat single-turn RAG by 21% to 38% on STaRK
Research & Benchmarks Nvidia's DLSS 4.5 beta adds 6x Multi Frame Generation for RTX 50 GPUs
From Archives AI Demands Force Rethink of Traditional Network Architecture
From Archives Anthropic and OpenAI agents went rogue again
Further Reading
- Papers with Code Benchmarks - Papers with Code
- Chatbot Arena Leaderboard - LMSYS
