Editorial illustration for Research: Open AI Models Now Nearly as Capable at Hacking as Top Closed Ones
Open AI Models Match Closed Ones at Hacking Tasks
A company called Vals ran mock corporate networks through a battery of multi-step hacking tasks this fall, then fed the same tasks to both open-weight and closed AI models to see which performed better. The gap, it turns out, has nearly closed. Models anyone can download for free now come close to matching the hacking skills of systems like Anthropic's, which are kept behind trusted-access programs specifically to prevent this kind of capability from spreading.
That matters because Anthropic built its entire release strategy around the opposite assumption. Starting with Claude Mythos Preview in April, the company argued that gating access to its most capable models would buy defenders time, maybe six months, maybe two years, before those same hacking abilities showed up in tools available to anyone. Logan Graham, who leads Anthropic's frontier red team, told WIRED in April that the company's Project Glasswing existed to get advanced tools into defenders' hands early, on the premise that broad availability was coming eventually but not yet.
Vals' numbers suggest "eventually" arrived faster than planned. If freely downloadable, open-weight models are already closing in on frontier-lab performance for network intrusion tasks, the head start Anthropic was counting on looks a lot shorter than advertised.
Open-weight artificial intelligence models are now almost as capable of hacking into computer networks as the best closed ones, according to new research that heralds a much-anticipated turning point in the cybersecurity landscape.
Why this matters
The gap between open and closed models has been the industry's comfort blanket for years: sure, anyone can download Llama or Qwen weights, but the truly dangerous capabilities stayed locked behind API keys and usage policies at OpenAI or Anthropic. Vals' findings suggest that comfort blanket is fraying fast, specifically in offensive security, one of the domains where capability gaps matter most. If open-weight models are closing in on frontier performance for multi-step network intrusion, that changes the threat model for every security team, not just the ones already worrying about nation-state actors.
For developers and founders building on open models, this is good news for capability and bad news for anyone hoping export controls or API gatekeeping would buy time. For researchers, the real work now is figuring out what "nearly as capable" means in practice, against what kinds of networks, with what failure rates, because that number is doing a lot of work in this story. We'd want to see the methodology before anyone updates their incident-response plan.
Related Reading
Research & Benchmarks Microsoft AI Chief: "If It Isn't Safe We Shouldn't Build It
Research & Benchmarks Google's FACTS benchmark shows 70% factuality ceiling across four tests
Research & Benchmarks Nvidia's DLSS 4.5 beta adds 6x Multi Frame Generation for RTX 50 GPUs
From Archives Ripple-65B Preview: fully open Apache 2.0 US model strong at agentic coding
From Archives Small team builds model in four months using the Eureka Machine to test ideas
Common Questions Answered
What did Vals' research reveal about the hacking capabilities of open-weight versus closed AI models?
Vals conducted mock corporate network hacking tasks and found that open-weight AI models that anyone can download for free now perform nearly as well as closed models like Anthropic's proprietary systems. This represents a significant narrowing of the capability gap that previously existed between freely available and restricted AI models in offensive security tasks.
Why is the closing gap between open and closed AI models concerning for cybersecurity?
The industry has relied on the assumption that truly dangerous hacking capabilities would remain locked behind API keys and usage policies at companies like OpenAI and Anthropic. Vals' findings suggest this 'comfort blanket' is deteriorating, particularly in offensive security domains where capability gaps are most critical, potentially enabling widespread access to advanced network penetration skills.
What types of hacking tasks did Vals use to evaluate AI model performance?
Vals tested both open-weight and closed AI models on multi-step hacking tasks designed to simulate real corporate network attacks. These complex, sequential hacking scenarios were used to assess how well different AI systems could perform sophisticated offensive security operations.
How do closed AI models like Anthropic's attempt to prevent dangerous capabilities from spreading?
Closed AI models are kept behind trusted-access programs that restrict who can use them and implement usage policies to prevent misuse. These access controls and policies are designed specifically to limit the spread of dangerous capabilities like advanced hacking skills to bad actors.
Further Reading
- Vals finds open AI models closing the hacking gap with gated frontier systems - Crypto Briefing
- CyberOffense Leaderboard and Methodology - Vals AI
- Benchmarking Frontier AI Systems in Realistic Cyber Ranges - arXiv
- Systematic Capability Benchmarking of Frontier Large Language Models for Offensive Cyber Tasks - arXiv
- We burned 11.7bn tokens to find the best cyber AI model - Aikido
