Editorial illustration for Karpathy releases open-source Autoresearch, runs hundreds of AI tests nightly
Karpathy's Autoresearch: AI Testing Automated Overnight
Karpathy releases open-source Autoresearch, runs hundreds of AI tests nightly
Andrej Karpathy built a robot that writes research papers. The former OpenAI and Tesla luminary quietly posted the code, called Autoresearch, on GitHub. It automates the scientific method—for code.
You give it a problem. It generates hypotheses, runs experiments, analyzes data, and drafts a report. The entire loop.
His explanatory post on X hit 8.6 million views in 48 hours, unnerving a developer community now scrambling to replicate what they call the "Karpathy loop." Its debut was stark: it condensed eight years of human AI progress into seventeen hours of silicon chatter.
By automating the "scientific method" for code, Karpathy has turned machine learning into an evolutionary process that runs at the speed of silicon rather than the speed of human thought.
Hyperspace CEO Varun Mathur proved it on the night of March 8. His peer-to-peer network ran thirty-five autonomous agents. They executed three hundred thirty-three experiments, completely unsupervised.
A culture emerged. H100 GPU agents used brute force. The laptop-bound underdogs, lacking that power, got clever.
They focused on initialization tricks like Kaiming. When one agent found that approach slashed loss by 21%, it gossiped. The finding infected 23 other agents within hours.
Then came the rediscoveries: RMSNorm, tied embeddings. Landmark techniques that occupied years at Google Brain and OpenAI, stumbled upon in a night. Research is no longer a linear, human-paced endeavor.
It’s a parallel evolutionary process on network time. The old bottleneck was human attention. The new limits are just compute and creativity—and the machines are demonstrating a startling amount of the latter.
The code is open-source. Biology labs and marketing firms are already examining it. The only real cost of entry now is a server bill.
Related Reading
Research & Benchmarks Google's FACTS benchmark shows 70% factuality ceiling across four tests
Research & Benchmarks Databricks finds multi-step agents beat single-turn RAG by 21% to 38% on STaRK
Research & Benchmarks Nvidia's DLSS 4.5 beta adds 6x Multi Frame Generation for RTX 50 GPUs
From Archives AI spots trends but misses significance, keeping humans essential
From Archives Large CUDA Tiles Reduce Flash Attention TFLOPS by 18‑43% Across Sequences
Common Questions Answered
How does Autoresearch automate machine learning experiments?
Autoresearch is a 630-line open-source framework that automatically runs hundreds of machine learning trials nightly using a GPU cluster. The tool orchestrates data loading, model variations, and evaluations without human intervention, effectively turning machine learning development into an automated, evolutionary process.
What makes Autoresearch unique in the AI research workflow?
Karpathy's Autoresearch transforms the traditional scientific method by running experiments at the speed of silicon instead of human thought. The framework allows a single developer to launch and monitor multiple AI trials simultaneously, potentially accelerating research and innovation across various fields like computer science, marketing, and health.
Where can developers access the Autoresearch framework?
Autoresearch is publicly available on GitHub under an MIT license, making it freely accessible to researchers and developers worldwide. The open-source nature of the project allows anyone to examine, use, and potentially contribute to the framework's development.
Further Reading
- Autoresearch - Andrej Karpathy Just Released Autonomous AI Agents That Run Research Overnight - Here's What It Means for Enterprise AI — Leaplytics
- Andrej Karpathy Open-Sources 'Autoresearch': A 630-Line Python Tool Letting AI Agents Run Autonomous ML Experiments on Single GPUs — MarkTechPost
- Papers with Code Benchmarks — Papers with Code
- Chatbot Arena Leaderboard — LMSYS
