Editorial illustration for Study: 6.7% of Deepfake Requests on Hugging Face Targeted Children
Deepfake Tool Abuse: 6.7% Target Children on Hugging Face
Study: 6.7% of Deepfake Requests on Hugging Face Targeted Children
AI Forensics, a European nonprofit that audits algorithms, published a report Tuesday examining nine of the most-used image editing tools on Hugging Face, the open-source AI platform valued in the billions and best known as a repository for machine learning models and datasets. Researchers found seven of the nine tools could take a clothed photo of a woman and generate a topless version with minimal effort. To dig further, the group built its own decoy image-editing tools on the platform, ones designed to produce no actual output, and logged what users sent them over the course of a week. More than 1,000 prompts and images came through.
The findings arrive as governments start moving against this kind of software. US law enforcement has seized deepfake hosting sites in recent months, and both the EU and UK have proposed banning "nudify" apps outright by year's end. Hugging Face, meanwhile, functions as an open marketplace where anyone can upload or run these models with little oversight. AI Forensics' numbers on who ends up targeted, and how often, make clear why regulators are paying attention now.
The open-source AI platform Hugging Face—a repository of AI models and datasets, which has been valued in the billions—has a widespread problem with nonconsensual deepfakes, according to a new report published Tuesday by the European nonprofit AI Forensics.
Why this matters
Hugging Face has built its reputation on being the open, developer-friendly alternative to closed AI shops, and that openness is exactly what's letting this happen. AI Forensics found Spaces on the platform generating nonconsensual intimate images, with 6.7 percent of sexual requests aimed at apparent children. That's not a hypothetical misuse case buried in terms of service. It's an active pipeline, running on infrastructure that developers and researchers rely on daily for legitimate model hosting.
For anyone building on Hugging Face, this is a moderation failure with legal exposure attached, not just a PR problem for the company. The EU and UK are drafting bans on "nudify" apps this year, and US officials have already started seizing hosting sites. If Hugging Face doesn't get ahead of enforcement, the platform itself becomes a target, and that risk trickles down to every legitimate project hosted alongside these Spaces.
Open-source AI's biggest asset, low barriers to publishing, is also its liability. Paul Bouchaud's research suggests the industry has been treating that tradeoff as someone else's problem for too long.
Related Reading
Open Source Trump cracks down on Anthropic after Amazon tip; staff largely foreign
Open Source SDOF Adds Two Defensive Layers via Intent Router and StateAwareDisp
Open Source D&B rebuilds 642 million‑business database after AI agents hit limits
From Archives Moonshot's Kimi K3 License Requires Separate Deal for USD 20M+ Revenue Firms
From Archives OpenAI Tests Hacking Capabilities of GPT‑5.6 Sol and Newer Models
Common Questions Answered
What did AI Forensics find when examining image editing tools on Hugging Face?
AI Forensics discovered that seven out of nine most-used image editing tools on Hugging Face could generate nonconsensual intimate images from clothed photos with minimal effort. The European nonprofit's audit revealed a widespread problem with deepfake creation on the open-source AI platform.
What percentage of deepfake requests on Hugging Face targeted children according to the study?
AI Forensics found that 6.7 percent of sexual deepfake requests on Hugging Face were aimed at apparent children. This represents an active pipeline of nonconsensual intimate image generation targeting minors on the platform.
How did AI Forensics conduct its investigation into nonconsensual deepfakes on Hugging Face?
Beyond examining nine existing image editing tools on the platform, AI Forensics built its own decoy image-editing tools on Hugging Face to gather additional data about deepfake request patterns. This methodology allowed researchers to document the scope and nature of nonconsensual intimate image generation occurring on the platform.
Why does Hugging Face's open-source model contribute to the deepfake problem?
Hugging Face has built its reputation as an open, developer-friendly alternative to closed AI platforms, but this openness is enabling the creation and distribution of nonconsensual deepfakes. The platform's accessibility and lack of sufficient safeguards allow bad actors to easily access and deploy tools for generating intimate images without consent.
Further Reading
- Hugging Face Has a Deepfake Nudes Problem - WIRED
- 'Deepfake abuse is abuse,' UNICEF warns - UN News
- A Marketplace for AI-Generated Adult Content and Deepfakes - arXiv
- the rise of accessible non-consensual deepfake image generators - arXiv
- Why is Hugging Face hosting tools to make deepfake porn of celebrities? - Transformer
