Editorial illustration for OpenAI Confronted Unexpected AI-Generated Erotica Problem in 2021
OpenAI's Wild 2021: When AI Generated Unexpected Erotica
Former OpenAI staffer details 2021 AI-generated erotica crisis
OpenAI likes to talk about preventing AI apocalypse. In 2021, its more immediate problem was AI-generated smut.
A new monitoring system flipped on that spring. It showed a river of traffic the company hadn't fully tracked. One major customer, a choose-your-own-adventure text game, was a big source. People were using it to make dirty stories.
Steven Adler had just taken over product safety. His team saw it and decided they had to act. This wasn't a theoretical risk. It was happening.
In that piece, you write that in the spring of 2021, your team discovered a crisis related to erotic content using AI. Can you tell us a little bit about that finding? So in the spring of 2021, I had recently become responsible for product safety at OpenAI.
As WIRED reported at the time, when we had a new monitoring system come online we discovered that there was a large undercurrent of traffic that we felt compelled to do something about. One of our prominent customers, they were essentially a choose-your-own-adventure text game.
The episode is a blunt object lesson. Before you worry about superintelligence, you have to stop your models from becoming automated erotica pens. Every AI company since has walked this same tightrope. They build systems capable of writing anything, then scramble to build filters that stop the things they don't want written.
Adler's account makes it clear the response was reactive. The monitoring found the problem. Then the safety team dealt with it. It shows how these companies are often flying blind, discovering misuse only after it's widespread.
This specific 2021 fix is now a historical footnote. The core tension isn't. The same pattern repeats with image generators, voice clones, and video tools.
The technology always outruns the guardrails. The first thing a significant number of people do with a new creative machine is try to get it to make something sexy, or violent, or both.
OpenAI solved its text erotica issue. The next company will face the same thing with a new medium. It's a cycle.
The real crisis isn't any single event. It's the permanent, exhausting game of whack-a-mole.
Further Reading
Related Reading
Browse all OpenAI news →
LLMs & Generative AI ChatGPT's 'Nerdy' tweak rewards goblin metaphors in answers, study finds
LLMs & Generative AI Google tests visual 'magazine-style' UI for Gemini 3 Pro users
LLMs & Generative AI AI Engineers Face Rising Costs, Need New Strategies for Efficiency
Related Topic OpenAI hires 630 ex-Meta staff as ChatGPT memory may turn data into ads
Business & Startups
Related Topic Sen. Warren urges Trump admin to expand AI credit after OpenAI letter
Business & Startups
From Archives Zomato MCP lets users order food via terminal with AI-powered commands
From Archives Gemini for TV arrives on Google TV Streamer; needs HDMI, HDCP 1.3+ and internet
From Archives ChatGPT API favors Wikipedia and obscure German sites, unlike web UI
From Archives OpenAI predicts modest AI finds by 2026, breakthroughs by 2028 as costs fall
Common Questions Answered
How did OpenAI first discover the issue of AI-generated erotic content in 2021?
OpenAI's product safety team uncovered a large undercurrent of erotic traffic when a new monitoring system came online in the spring of 2021. The discovery prompted internal concern about the potential risks of unrestricted AI content generation.
What challenges did OpenAI's product safety team face with AI-generated content in 2021?
The team confronted unexpected issues related to erotic material produced by their language models, which revealed complex content moderation challenges. This incident highlighted the potential risks of unrestricted creative potential in artificial intelligence systems.
Why was the AI-generated erotic content considered a significant problem for OpenAI?
The surge of sexual content generation was substantial enough that OpenAI's product safety team felt compelled to take immediate action. The incident underscored the potential risks lurking within AI systems and the need for robust content moderation strategies.
Further Reading
- OpenAI's Sam Altman says ChatGPT will add erotica for adult users — Axios
- OpenAI's Sam Altman clarifies "adult model" plans amid erotica backlash — Storyboard18
- Papers with Code - Latest NLP Research — Papers with Code
- Hugging Face Daily Papers — Hugging Face
- ArXiv CS.CL (Computation and Language) — ArXiv
