Editorial illustration for Open Models Offer Benefits, But an Extractive Approach Raises Safety Risks
Open Source AI Models: Safety Risks of Unrestricted Access
Open Models Offer Benefits, But an Extractive Approach Raises Safety Risks
Anthropic put a number on its worry last month: GLM-5.3, the model Chinese company Z.ai released in August, poses what the company's report calls a major cyberattack risk. The reason isn't the model's capability alone. It's that anyone can download it, strip out its guardrails, and run it on their own hardware. That single fact reopens a fight AI companies have been having for years, over whether the most capable models should be released openly or kept behind the walls of a handful of labs.
The argument has an obvious financial undertow. Nvidia's Jensen Huang wants Americans buying more chips to run open models at home, so he pushes hard for openness. Dario Amodei runs Anthropic, a company that open, cut-rate competitors could gut, so he pushes caution.
Money aside, the two camps are also arguing about something harder to pin down: what AI actually is, how dangerous it gets as it improves, and who should get to control that trajectory. Where someone lands on open models, it turns out, has a lot to do with where they land on AI generally.
There are a range of ways to handle open models between banning them entirely and releasing them willy-nilly. But any approach that a government or a society might take still involves trade-offs between safety and democratization. Deciding whether to support or limit the production of open models requires understanding those risks and benefits in depth.
Why this matters
GLM-5.3 and the Anthropic report point to a problem we keep underestimating: the open-closed split isn't really about safety architecture, it's about who captures the value once a model ships. An "extractive approach" means a lab releases weights, lets the ecosystem build on top, and walks away from the harder work of managing what happens next. That's a business decision dressed up as an openness debate.
For developers building on open weights, the Hugging Face hack and the OpenAI agent exploit should be the actual data point, not the branding. Closed models aren't demonstrably safer right now; they're just differently exposed. Founders betting on open models for cost or control need to budget for the fact that nobody upstream is patching misuse the way a closed-model vendor might.
Researchers should push past the open-vs-closed framing entirely and ask who's accountable for a model's behavior six months after release, because that's the question GLM-5.3 actually raises. Watch whether Zhipu or any other open-weight lab responds with real mitigation, not just a statement.
Related Reading
Open Source D&B rebuilds 642 million‑business database after AI agents hit limits
Open Source Box CEO Aaron Levie Says AI Will Drive "Biggest Cybersecurity Upgrade
Open Source Study: 6.7% of Deepfake Requests on Hugging Face Targeted Children
From Archives Microsoft’s New AI Chip Promises 40% More Tokens Per Megawatt Than GPUs
From Archives OrcaCyber Zero 1.5 Model Boasts Top-Tier Cyber Scores at USD 3 Per 1M Tokens
Common Questions Answered
Why does Anthropic consider GLM-5.3 a major cyberattack risk despite its capabilities?
Anthropic's concern with GLM-5.3 stems not just from the model's capabilities, but from the fact that anyone can download it and strip out its guardrails to run on their own hardware. This ability to remove safety mechanisms creates significant security vulnerabilities that wouldn't exist if the model remained behind controlled lab walls.
What trade-offs exist between safety and democratization when deciding on open model releases?
Governments and societies must weigh the benefits of democratizing AI access against potential safety risks when determining open model policy. Any approach between completely banning open models and releasing them without restrictions involves inherent trade-offs that require careful consideration of both the risks and benefits involved.
What is an 'extractive approach' in the context of open model releases?
An extractive approach occurs when a lab releases model weights, allows the ecosystem to build on top of them, and then withdraws from managing the consequences of that release. This represents a business decision that prioritizes capturing initial value while avoiding the harder work of overseeing downstream applications and safety management.
How does the open-closed model split differ from a traditional safety architecture debate?
Rather than being primarily about safety architecture differences, the open-closed split is fundamentally about who captures value once a model is released. The framing as an openness debate often obscures the underlying business incentives and decisions that drive whether companies choose to release model weights publicly or keep them proprietary.
Further Reading
- Papers with Code - Latest NLP Research - Papers with Code
- Hugging Face Daily Papers - Hugging Face
- ArXiv CS.CL (Computation and Language) - ArXiv
