Editorial illustration for FineBooks Aims to Fix Old OCR Text for AI Training at Scale
FineBooks Fixes Corrupted OCR Text for AI Training
Public-domain books are a favorite training source for open language models, but the text libraries scanned decades ago is often garbage. Optical character recognition from that era mangled letters, dropped words, and turned clean pages into noisy data. The Talkie project measured just how bad the fallout gets: a model trained on that old OCR output learned at only 30 percent the efficiency of one trained on human transcriptions of the same material.
That gap is what FineBooks set out to close. The project, a joint effort between Hugging Face and EleutherAI, put 14 open-weight OCR models through a test against more than 2,000 pages pulled from historical books. The goal wasn't academic-grade transcription. It was figuring out whether today's OCR tools can produce text clean enough to actually train AI models on, at a price and speed that make sense for processing libraries at scale rather than a few hundred pages in a lab.
The results point to a specific tradeoff between accuracy, cost, and model size, one that runs counter to the usual assumption that bigger models win.
FineBooks, a collaboration between Hugging Face and EleutherAI, tested whether current open-source OCR models can solve this problem. The team ran 14 open-weights models on 2,165 historical book pages and published the results as a leaderboard. The best models hit character accuracy above 97 percent at less than two dollars per thousand pages.
Why this matters
The benchmark results should reshuffle priorities for anyone building training pipelines from historical text. A model hitting 97 percent character accuracy for under two dollars per thousand pages changes the math on digitizing public-domain archives that have sat untouched because the cost or quality tradeoffs looked bad. That smaller models beat larger ones on this task is the real signal here, not a footnote.
It suggests teams have been defaulting to scale as a proxy for quality when the actual bottleneck was task-specific tuning on messy scans, not parameter count. For researchers at EleutherAI and Hugging Face, this feeds directly into corpora like Common Pile, where clean text at volume determines whether "openly licensed" also means "usable." For founders building on top of these datasets, it's worth asking which OCR model your pipeline actually uses and whether you've benchmarked it against FineBooks' list, because "we used OCR" is not the same claim as "we used OCR that works." Watch whether Common Pile's next release incorporates these findings.
Related Reading
Research & Benchmarks Google's FACTS benchmark shows 70% factuality ceiling across four tests
Research & Benchmarks Databricks finds multi-step agents beat single-turn RAG by 21% to 38% on STaRK
Research & Benchmarks Nvidia's DLSS 4.5 beta adds 6x Multi Frame Generation for RTX 50 GPUs
From Archives Hidden PDF Text Can Steal Data via Atlassian's AI Agent Rovo
From Archives Deepmind WeatherNext Cuts Cyclone Forecast Error to 230-Kilometer Average
Common Questions Answered
Why does old OCR text negatively impact language model training efficiency?
Historical OCR systems from decades ago produced poor quality scans that mangled letters, dropped words, and created noisy data. According to the Talkie project, models trained on this degraded OCR output learned at only 30 percent the efficiency of models trained on human transcriptions of the same material, creating a significant performance gap.
What are the key findings from FineBooks' testing of open-source OCR models?
FineBooks, a collaboration between Hugging Face and EleutherAI, tested 14 open-weights OCR models on 2,165 historical book pages and published results as a leaderboard. The best performing models achieved character accuracy above 97 percent at a cost of less than two dollars per thousand pages, making it economically viable to digitize public-domain archives.
How does the cost-to-accuracy ratio change the economics of digitizing public-domain archives?
With top OCR models now achieving 97 percent character accuracy for under two dollars per thousand pages, the cost and quality tradeoffs for digitizing historical texts have fundamentally improved. This breakthrough makes it economically feasible to digitize public-domain archives that were previously considered too expensive or low-quality to process at scale.
Why is the finding that smaller OCR models outperformed larger ones significant for training pipelines?
The FineBooks results showed that smaller models beat larger ones on the OCR correction task, suggesting that teams have been incorrectly defaulting to scale as a proxy for quality. This insight should reshape priorities for anyone building training pipelines from historical text, as it indicates that model size alone is not the determining factor for OCR accuracy on this task.
Further Reading
- FineBooks: are open OCR models good enough to unlock historical knowledge at scale? - Hugging Face
- OCR Accuracy Explained: How to Improve It - LlamaIndex
- What Is Fine-Tuning For Documents? - LlamaIndex
- GitHub - allenai/olmocr: Toolkit for linearizing PDFs for LLM datasets/training - AllenAI
- A (free) 200 Page LLM Training Playbook, epic OCR models, SAM 3? - YouTube
