Karpathy releases nanochat: 8,000-line PyTorch LLM roadmap (Oct 13 2025)
On October 13, 2025 Andrej Karpathy pushed a new repo called nanochat. It’s a PyTorch-based language model that squeezes most of its logic into about 8,000 lines. The code is open source, and the guide that comes with it walks you through training from scratch and wrapping the result into a private AI that you can have up and running in a few hours.
The trick is that the codebase is intentionally tiny - more a roadmap than a black-box library. Even though it’s short, it still touches every part of the pipeline: data preprocessing, model architecture, the training loop, and an inference API. The claim is straightforward: if you have a modest GPU, you should be able to follow the steps and end up with a working chatbot without splurging on huge compute.
The original tutorial is titled “Build ChatGPT Clone with Andrej Karpathy’s nanochat,” which hints at the goal - mimic a well-known conversational system using a lightweight, transparent codebase. Whether this scales beyond hobby projects is still unclear, but it does give developers a concrete foothold if they’re curious about building their own language models.
Launched on October 13, 2025, Karpathy’s nanochat project is an open-source LLM coded in roughly 8,000 lines of PyTorch. It gives you a straightforward roadmap on how to train a language model from scratch and make your own private AI in a couple of hours. In this article, we will talk about the newly released nanochat and how to properly set it up for the training step by step.
The nanochat repository provides a full-stack pipeline to train a minimal ChatGPT clone. It takes care of everything from tokenization to the end web user interface. This system is a successor to the previous nanoGPT.
It introduces key features such as supervised fine-tuning (SFT), reinforcement learning (RL), and enhanced inference. The project has a number of significant components. It incorporates a new Rust-built tokenizer for high performance.
The training pipeline employs quality data such as FineWeb-EDU for pretraining. It also employs specialized data such as SmolTalk and GSM8K for post-training fine-tuning.
Karpathy’s nanochat shows up as a tiny, open-source LLM written in about 8,000 lines of PyTorch. The repo promises a straightforward roadmap: you could train a model from scratch and have a private AI up and running in a few hours, all for roughly $100. The accompanying article walks you through the setup and training steps, which makes the code feel like a realistic entry point for hobbyists or small teams.
Still, the claim of “ChatGPT-like” performance raises a few eyebrows - the model’s size, the quality of its training data, and how fast it runs on modest hardware are all open questions. The write-up skips any detail on the compute needed and doesn’t give benchmark numbers against bigger commercial systems. So, while the code is undeniably easy to get into, it’s unclear whether the resulting assistant will be fluent or reliable enough for everyday conversation.
If you’re thinking about trying it, weigh the appeal of an 8-kiloline implementation against the unknowns around performance, scaling and long-term upkeep. I found the step-by-step guide handy, but be ready for some debugging and tuning along the way.
Further Reading
- Andrej Karpathy Releases 'nanochat': A Minimal, End-to-End ChatGPT-Style Pipeline You Can Train in 4 Hours for $100 - MarkTechPost
- nanochat - Simon Willison's Weblog - Simon Willison's Weblog
- Nanochat: Create your own LLM, train it, and get it running on your PC for $100 - Red Hot Cyber
More in LLMs & Generative AI
Google pushes Gemini rollout to 2026, delays Android Assistant swap
Why does this matter now? For years, Android users have expected a smooth handoff from the familiar...
OpenAI now adds tone controls and streamlined email editing to ChatGPT
OpenAI just rolled out a set of tweaks that feel oddly personal for a chatbot. While the core model...
Google releases FunctionGemma, a tiny model for natural‑language mobile control
Here's the thing: mobile phones are the most common computing platform, yet most AI assistants...
Chinese scammers used AI images for refund fraud; police held buyer eight days
When a shopper in Shanghai posted a video of a defective product, the footage looked...
Gemini pulls Google Calendar data to auto‑suggest meeting times in email replies
Why does this matter? In a flood of AI shortcuts that promise to shave minutes off daily tasks,...
Common Questions Answered
What is the primary goal of Karpathy's nanochat project released on October 13, 2025?
The primary goal of nanochat is to serve as a clear and compact roadmap for training a language model from scratch using PyTorch. It is designed as an open-source project with roughly 8,000 lines of code to demystify the process, enabling users to create a private AI in a couple of hours rather than being a complex black-box library.
What does the nanochat repository provide for training a language model?
The nanochat repository provides a full-stack pipeline to train a minimal ChatGPT clone from the ground up. It includes an accompanying guide that offers step-by-step instructions for the entire process, from initial setup to completing the training, making it accessible for hobbyists and small teams.
According to the article, what is the estimated cost and time to get a private AI running with nanochat?
The project promises that users can spin up a private AI in just a few hours for an estimated cost of about $100. This makes nanochat a practical and affordable entry point for individuals or small groups interested in developing their own language models without significant financial investment.
What programming language and approximate code size is the nanochat LLM implemented in?
Nanochat is implemented in PyTorch and consists of roughly 8,000 lines of code. This deliberately compact codebase is intended to be easily understandable and serve as an educational roadmap for building a language model.