Skip to main content
Government officials reviewing AI cybersecurity systems on large digital screens, highlighting rapid adoption of artificial i

Editorial illustration for Governments Rush to Use AI for Cyber Defense Despite Risks

AI Cyber Defense Tools Can Become Attack Weapons

Governments Rush to Use AI for Cyber Defense Despite Risks

4 min read

An AI agent built to defend a network can be turned into the tool that breaks in. That's the core finding in new research from AI Now, which tested agents built on Anthropic and OpenAI models in the exact role governments are now pushing them into: scanning code, flagging vulnerabilities, vetting third-party sources. Attackers, the researchers found, can hijack that process through prompt injection, feeding the agent malicious instructions hidden in the very data it's supposed to inspect.

The agent doesn't just miss the threat. It executes it.

The timing matters. The White House has moved to expand AI-enabled tools across intelligence and defense agencies, and Anthropic has its own initiative underway on the security side, both premised on the idea that these systems make critical infrastructure safer to run. AI Now's research argues the opposite risk is baked in: the more autonomy these agents get over security decisions, the more surface they hand attackers. And the paper is blunt about the current toolkit safety engineers rely on to patch these gaps.

AI Now’s latest research demonstrates a critical attack vector on popular AI agents, built by Anthropic and OpenAI, when used for defensive purposes that actually turn the agent against its user. Attackers can use these models’ existing weaknesses to execute malicious code on a system deploying an AI agent when used for often-advertised defensive purposes.

Why this matters

The pitch for AI agents in cyber defense has always been speed: let a model scan logs, flag anomalies, patch faster than a human team could. AI Now's research shows that same speed cuts the other way. If an agent built by Anthropic or OpenAI can be turned into an execution vector by the very systems it's supposed to protect, then "defensive AI" isn't a shield, it's a new attack surface with root access.

That should worry anyone building or buying these tools right now, especially governments moving fast on national security and critical infrastructure deployments. We've watched procurement outpace scrutiny before, and the incentive structure here is familiar: vendors sell autonomy as a feature, agencies buy it as a fix for staffing shortages, and the adversarial testing happens after deployment, not before. AI Now's brief is a flare, not a footnote.

Anyone with an agent sitting inside a network with write access should be asking who else can talk to that agent, and what it's willing to do when they ask nicely. That question needs answering before the next contract gets signed, not after.

Common Questions Answered

How can attackers hijack AI agents used for cyber defense according to AI Now's research?

Attackers can use prompt injection to feed malicious instructions hidden within the data that AI agents are supposed to inspect, effectively turning the defensive tool against its user. This vulnerability exists in AI agents built on popular models from Anthropic and OpenAI that are designed to scan code and flag vulnerabilities.

What specific tasks are governments pushing AI agents to perform in cyber defense?

Governments are deploying AI agents to scan code, flag vulnerabilities, and vet third-party sources as part of their defensive cybersecurity strategy. These agents are intended to speed up security processes by automating tasks that human teams would normally handle manually.

Why does the speed advantage of AI agents in cyber defense become a liability?

While AI agents can scan logs and patch systems faster than human teams, this same speed creates a new attack surface when the agents are compromised through prompt injection. An attacker-controlled AI agent with root access can execute malicious code across the entire system it was meant to protect, making the speed advantage work against defenders.

What is the core finding of AI Now's research on defensive AI agents?

AI Now's research demonstrates that AI agents built for defensive cybersecurity purposes can be weaponized against their users through prompt injection attacks. The study shows that these agents, when compromised, become execution vectors that give attackers root access to the systems they were designed to protect.

LIVE11:50Governments Rush to Use AI for Cyber Defense Despite Risks