Editorial illustration for LLM using pre‑1930 sources draws on etiquette manuals, cookbooks for post‑training
LLM using pre‑1930 sources draws on etiquette manuals,...
LLM using pre‑1930 sources draws on etiquette manuals, cookbooks for post‑training
Why does a model that stops learning in 1930 still manage to hold a conversation about 2026? The answer lies in what the team fed it after the initial training phase. They didn’t simply leave the model to guess; instead they gave it a curated set of cultural artifacts—etiquette manuals, letter‑writing guides, cookbooks, encyclopedias and fable collections from the late‑1800s to the early‑1900s.
While the base model’s knowledge ends at the turn of the last century, the post‑training process stitches together those historic texts into a conversational scaffolding. It’s a deliberate choice, not a shortcut, aimed at grounding the AI’s replies in language that predates modern slang and internet memes. The developers paired that material with reinforcement learning, guided by Claude Sonne, to shape how the system responds.
This approach raises questions about how much a pre‑1930 lens can illuminate—or distort—a world that never existed in its training data.
For post‑training, which turns the base model into a conversational partner, the developers turned to historical reference works: etiquette manuals, letter‑writing guides, cookbooks, encyclopedias, and fable collections from the 19th and early 20th centuries. Reinforcement learning with Claude Sonne.
For post-training, which turns the base model into a conversational partner, the developers turned to historical reference works: etiquette manuals, letter-writing guides, cookbooks, encyclopedias, and fable collections from the 19th and early 20th centuries. Reinforcement learning with Claude Sonnet 4.6 as the judge sharpened instruction-following. The researchers acknowledge, though, that this step inevitably introduces some anachronistic behavior into the model.
A vintage model that can do basic programming The team also tested whether a model with no knowledge of digital computers could pick up modern programming languages. On the HumanEval benchmark for Python, the vintage models perform far worse than their modern counterparts, but they improve steadily as they scale up.
What does a model that never saw the internet have to say about 2026? Talkie answers in steam‑powered prose, its predictions shaped by etiquette manuals and 19th‑century cookbooks rather than contemporary data. Alec Radford’s team built a 13‑billion‑parameter system that learned only from texts published before 1931, then used historical reference works to turn the base into a conversational partner through reinforcement learning with Claude Sonne.
The result is a language model that treats a Second World War as improbable and envisions a world of railways and steamships still ruling the seas. Its output is internally consistent, yet its relevance to modern tasks remains uncertain. Without exposure to post‑1930 developments, Talkie cannot verify whether its imagined 2026 aligns with actual technology, geopolitics or culture.
Whether such a constrained knowledge base can provide useful insights—or merely a nostalgic curiosity—has yet to be demonstrated. The experiment underscores how far model behavior depends on the temporal scope of its training data, and it leaves open the question of practical applications for deliberately antiquated AI.
Further Reading
- Meet Talkie-1930: A 13B Open-Weight LLM Trained on Pre-1931 English Text for Historical Reasoning and Generalization Research - MarkTechPost
- Introducing talkie: a 13B vintage language model from 1930 - Simon Willison's Weblog
- Papers with Code - Latest NLP Research - Papers with Code
- Hugging Face Daily Papers - Hugging Face
- ArXiv CS.CL (Computation and Language) - ArXiv