Editorial illustration for AI Model Claude 3.7 Reproduces 3,000 Words from Classic Fantasy Novels
Claude 3.7 Reproduces Entire Book Passages Verbatim
RECAP tool shows Claude 3.7 reproduces ~3,000 words from The Hobbit and Harry Potter
AI models are supposed to summarize and synthesize, not recite. A new test suggests they might be doing a lot more of the latter.
Researchers have built a tool, called RECAP, to force large language models to cough up what they've memorized. The first results from testing Claude 3.7 are not subtle. The model can reproduce roughly 3,000 words from "The Hobbit" and the first "Harry Potter" book, often nearly verbatim.
This isn't a quirky party trick. It is a direct line into the model's training data, a demonstration that these systems retain far more raw text than anyone assumed. The legal and creative consequences are messy, expensive, and unresolved.
In testing, RECAP was able to reconstruct large portions of books like "The Hobbit" and "Harry Potter" with striking accuracy.
The scale of the regurgitation is the real problem. Earlier methods found 75 passages. RECAP found 3,000. That difference suggests we've been dramatically underestimating how much copyrighted material these models have on file, ready to play back.
The EchoTrace benchmark made the test broad, using everything from old classics to modern bestsellers. The model reproduced text from almost all of them. The only exceptions were the books published after its training cut-off. This proves the source is the training data, not some emergent linguistic genius.
For copyright lawyers, this is a field day. For authors and publishers, it's a nightmare. An AI that can reconstruct thousands of words from a novel is not learning style or concepts.
It is performing a kind of compressed memorization. The argument that this is "transformative" use just got a lot harder to make.
We built tools to create new things. They seem better at remembering old ones.
Further Reading
- Anthropic's Claude 3.7 caught regurgitating entire chapters from The Hobbit in RECAP benchmark - Ars Technica
- Claude 3.7 memorizes and reproduces thousands of words from famous books, RECAP tool reveals - TechCrunch
- New benchmark exposes Claude 3.7's training data memorization with Hobbit and Potter excerpts - The Verge
- RECAP test: How Claude 3.7 reproduced 3,000 words from Tolkien and Rowling's works - Wired
Related Reading
Browse all Anthropic news →
Research & Benchmarks Google's FACTS benchmark shows 70% factuality ceiling across four tests
Research & Benchmarks Databricks finds multi-step agents beat single-turn RAG by 21% to 38% on STaRK
Research & Benchmarks Nvidia's DLSS 4.5 beta adds 6x Multi Frame Generation for RTX 50 GPUs
Related Topic Trump cracks down on Anthropic after Amazon tip; staff largely foreign
Open Source
Related Topic Anthropic says Claude writes >90% of its code; AI pause button urged
LLMs & Generative AI
From Archives ElevenLabs' Scribe v2 delivers real-time, negative-latency transcription
From Archives Meta's SPICE framework beats baselines, boosts math and general reasoning
From Archives Anthropic projects revenue jump from USD 4.7B to USD 70B by 2028, API to drive 80%+
Business & Startups
From Archives Anthropic's Claude reduces SOC investigations from 5 hrs to 7 min, 95% accuracy
LLMs & Generative AI
Common Questions Answered
How many passages did Claude 3.7 reproduce from the first Harry Potter book using the RECAP method?
Claude 3.7 generated approximately 3,000 passages from the first Harry Potter book using the RECAP technique. This is a dramatic increase compared to only 75 passages found by previous investigative methods.
What books were used in testing the RECAP tool's text reproduction capabilities?
The research team used the EchoTrace benchmark, which included 35 complete books comprising 15 public domain classics and 15 copyrighted books. Specific examples mentioned include classic fantasy novels like 'The Hobbit' and 'Harry Potter'.
What potential implications does the RECAP method have for intellectual property rights?
The RECAP tool's findings suggest that AI models like Claude 3.7 can reproduce substantial portions of copyrighted text with remarkable precision. These results could fundamentally challenge existing copyright frameworks and raise serious questions about data retention and intellectual property protection.
Further Reading
- New RECAP tool exposes just how much copyrighted text LLM's can regurgitate — The Decoder
- RECAP: Reproducing Copyrighted Data from LLMs Training with an Automated Pipeline — ChatPaper
- Papers with Code Benchmarks — Papers with Code
- Chatbot Arena Leaderboard — LMSYS
