Editorial illustration for LLM using pre‑1930 sources draws on etiquette manuals, cookbooks for post‑training
LLM using pre‑1930 sources draws on etiquette manuals,...
Some AI developers are bored with the present. They made a language model that thinks the world ended nearly a century ago.
This model wasn't trained on the internet. For the final stage of tuning, its creators fed it a diet of pre-1930s literature. Think etiquette guides, fables, and cookbooks.
The idea was to shape a conversational partner with the manners and worldview of a Victorian. They then used AI from Anthropic to refine its responses. The result is a chatbot stuck in a bygone era.
It knows how to address a duke but has no concept of a smartphone. The developers even wondered if this relic could learn modern coding. They tested it on Python problems.
Another headache is keeping knowledge from later eras out of the training data. A 1925 book might pick up an updated preface in a 1960 edition, library catalogs sometimes list the wrong publication date, and footnotes or commentary can be added to a historical text long after it was written.
On the coding test, the antique model was predictably bad. Yet as they made it bigger, it got better. This is the quiet revelation.
A mind built from cookbooks and letter-writing guides can learn to write a for-loop. The logic of a recipe is not so different from the logic of an algorithm. The structure of a fable mirrors the structure of a function.
The charm is in the deliberate flaws. It might suggest whale oil for your lamp. This isn't a mistake to be fixed.
It's the whole point. We asked for a voice from the past and we got one, complete with its period prejudices and peculiar wisdom.
It proves something simple but profound. The core scaffolding of human knowledge is timeless. It also proves something darker.
Every dataset is a choice, a filter that blinds the machine to certain truths. This one just happens to wear its blinders with old-fashioned style.
Related Reading
LLMs & Generative AI ChatGPT's 'Nerdy' tweak rewards goblin metaphors in answers, study finds
LLMs & Generative AI Google tests visual 'magazine-style' UI for Gemini 3 Pro users
LLMs & Generative AI AI Engineers Face Rising Costs, Need New Strategies for Efficiency
From Archives Alibaba's Metis agent cuts redundant AI tool calls to 2% and boosts accuracy
From Archives Qiushi Discovery Engine Enables Autonomous Science on Optical Platform
Common Questions Answered
What training sources were used for the post-training phase of this pre-1930s language model?
The model was trained on pre-1930s literature including etiquette guides, fables, and cookbooks during its final tuning stage. Rather than using internet data, the developers deliberately selected historical texts to shape a conversational partner with Victorian-era manners and worldview.
How does the model perform on coding tasks compared to modern language models?
The antique model performed predictably poorly on coding tests initially, but surprisingly improved as the model was scaled up in size. This demonstrated that the logical structure found in recipes and fables could transfer to programming concepts like for-loops and functions.
Why are the model's anachronistic suggestions considered a feature rather than a flaw?
The developers intentionally preserved the model's historical perspective, so suggestions like using whale oil for lamps are deliberate design choices rather than errors. These charming anachronisms are central to the project's goal of creating a conversational partner with authentic pre-1930s knowledge and values.
What is the key insight about the relationship between recipe logic and algorithm logic?
The creators discovered that the logical structure of recipes shares fundamental similarities with the logic of algorithms, and that the structure of fables mirrors the structure of functions. This revelation shows that historical texts contain transferable logical patterns that can help language models learn programming concepts.
Further Reading
- Meet Talkie-1930: A 13B Open-Weight LLM Trained on Pre-1931 English Text for Historical Reasoning and Generalization Research — MarkTechPost
- Introducing talkie: a 13B vintage language model from 1930 — Simon Willison's Weblog
- Papers with Code - Latest NLP Research — Papers with Code
- Hugging Face Daily Papers — Hugging Face
- ArXiv CS.CL (Computation and Language) — ArXiv
