AI vision pioneer aims to extend models from data to space understanding
The researcher who first gave machines the ability to recognize objects is now turning his attention to something bigger: space. After years of building vision models that can label cats, read street signs and sort millions of images, he argues that the next frontier isn’t more data—it’s the world itself. He points to the gap between what AI can do on a screen and what it can do in a room, on a street, or in orbit.
Why does that matter? Because the same algorithms that power chatbots and recommendation engines are being pressed into service for robotics, autonomous vehicles and satellite navigation. Yet every test run seems to hit a wall when the system must move from pixels to physics.
The scientist’s latest paper lays out the problem in stark terms, noting that while today’s top‑tier models dominate reading, writing and pattern‑finding tasks, they fall short when it comes to representing or interacting with the physical world.
*While current state‑of‑the‑art AI can excel at reading, writing, research, and pattern recognition in data, these same models bear fundamental limitations when representing or interacting with the physical world,* Li writes. Humans, on the other hand, seem to integrate perception and meaning seamle
"While current state‑of‑the‑art AI can excel at reading, writing, research, and pattern recognition in data, these same models bear fundamental limitations when representing or interacting with the physical world," Li writes. Humans, on the other hand, seem to integrate perception and meaning seamlessly. We don't just recognize a coffee mug, we instantly grasp its size, its weight, and where it sits in space.
That implicit spatial reasoning, says Li, is something AI still lacks entirely. The cognitive scaffold that made us intelligent Li traces the roots of intelligence back to the simplest perceptual loops. Long before animals could nurture offspring or communicate, they sensed and moved, starting a feedback cycle that eventually gave rise to thought itself.
That same ability underpins everything from driving a car to sketching a building or catching a ball. Words can describe these acts, but they can't reproduce the intuition behind them. "Thus, many scientists have conjectured that perception and action became the core loop driving the evolution of intelligence," Li notes.
How spatial insight powered human discovery Throughout history, Li writes, breakthroughs have often come from seeing the world (literally) differently: Eratosthenes calculated Earth's circumference using shadows cast in two Egyptian cities at the same moment.
Will machines ever navigate space as intuitively as animals? Li argues that the missing piece is an understanding of motion, distance and physical relationships, not more text. Current models excel at reading, writing and pattern recognition, yet they stumble when asked to represent or act within the physical world.
Humans, by contrast, fuse perception with meaning without conscious effort. The Stanford pioneer, co‑creator of ImageNet and founder of the startup World, sees this gap as the next frontier. Evolution granted a sense of place half a billion years ago; silicon may be catching up.
However, the path from algorithmic perception to embodied spatial reasoning remains unclear. Li’s essays acknowledge fundamental limitations, but offer no concrete roadmap for overcoming them. Whether future systems can truly become “creative partners” by mastering space is still an open question.
The ambition is clear, the challenges are real, and the outcome is uncertain for now.
Further Reading
- AI’s Next Five Years: LeCun Predicts a Physical-World Revolution - Klover.ai
- The Shape of AI to Come! Yann LeCun at AI Action Summit 2025 - AI Pursuit
- Meta AI Chief Yann LeCun: Human Intelligence Is Not General - Jason Howell Substack
- Yann LeCun on a vision to make AI systems learn and reason like humans - Meta AI Blog
Common Questions Answered
What limitation does Li identify in current state‑of‑the‑art AI models regarding interaction with the physical world?
Li points out that while modern AI excels at reading, writing, and pattern recognition in data, it fundamentally struggles to represent or act within the physical world. This limitation prevents models from understanding motion, distance, and spatial relationships needed for real‑world tasks.
How does Li compare human spatial reasoning to AI's capabilities in the article?
Li notes that humans seamlessly integrate perception with meaning, instantly grasping an object's size, weight, and position in space without conscious effort. In contrast, AI lacks this implicit spatial reasoning and cannot intuitively infer physical properties from visual inputs.
Why does Li argue that extending AI models from data to space understanding is more important than simply adding more data?
Li believes the next frontier is teaching machines to comprehend motion, distance, and physical relationships, which data alone cannot provide. By focusing on spatial understanding, AI can move beyond screen‑based tasks to operate effectively in rooms, streets, and orbit.
What is the significance of Li's role as co‑creator of ImageNet and founder of the startup World to his new focus on space understanding?
Li's work on ImageNet established foundational vision models that can label millions of images, demonstrating his expertise in visual perception. His startup World builds on that legacy, aiming to bridge the gap between image recognition and real‑world spatial reasoning.