Editorial illustration for Runway's open-source Praxis-1 turns video AI into a robot controller
Runway's Praxis-1 Uses Video AI to Control Robots
Runway's open-source Praxis-1 turns video AI into a robot controller
Runway built its name on text-to-video generation, the kind of tool marketing teams use to fake a commercial without hiring a camera crew. Now the company wants that same video know-how to steer actual robots. On release day, Runway put Praxis-1's weights and source code online under the Apache 2.0 license, meaning anyone, including competitors, can take the model and build commercial products on top of it with no royalties owed.
Praxis-1 started life as a video prediction model, the kind trained to guess how a scene will unfold frame by frame. Runway introduced it to select partners back in September before opening it up to everyone this week. The pitch is straightforward: feed the model a camera image, a text instruction like "pick up the cup," and the robot's current position, and it outputs the movement needed to get there. It keeps recalculating as new frames arrive, so the control loop updates in near real time rather than planning once and executing blind.
Runway published a research report alongside the launch laying out performance numbers and, notably, where the model still falls short.
Praxis-1 builds on a video model that learned how objects move and interact from ordinary footage. Runway then fine-tuned it with videos of human actions and robot data. This approach could reduce the need for expensive robot-specific datasets, which have traditionally been collected just to train these kinds of systems.
Why this matters
Runway putting Praxis-1's weights and code under Apache 2.0 is the real story here, not the robotics framing. A company that built its name on video generation is now claiming that same visual prediction engine can drive physical actuators, and it's handing the architecture to anyone who wants to pull it apart. For developers and founders in robotics, that's a cheaper starting point than training a world model from scratch, assuming the claims about adapting to new robot systems through additional training hold up outside Runway's own demos.
The admission that it still confuses similar-looking objects is worth sitting with. That's not a rounding error in a robot that's supposed to translate scene prediction into movement, it's a core perception failure. Standard Bots, Noble Machines, and Ultra have apparently been testing this since September, so there's already real-world mileage on it, which makes it harder to dismiss as a research toy.
We'd want to see those partners' failure logs before betting a product roadmap on it. Open weights invite scrutiny, and this model is going to get plenty.
Related Reading
Open Source Study: 6.7% of Deepfake Requests on Hugging Face Targeted Children
Open Source D&B rebuilds 642 million‑business database after AI agents hit limits
Open Source Axiom Math raises USD 200M Series A led by Menlo to keep AI slop out of math
From Archives Vercel Releases UI SDK With 50+ Behaviors for AI Coding Agents
From Archives Keyword Search Tied an 8B Embedding Model at Picking Agent Tools
Common Questions Answered
How does Runway's Praxis-1 model transition from video generation to robot control?
Praxis-1 started as a video prediction model trained on ordinary footage to understand how objects move and interact. Runway then fine-tuned this model using videos of human actions and robot data, enabling it to translate visual predictions into robot control commands. This approach leverages the same visual understanding capabilities that power Runway's text-to-video generation tools.
What is the significance of Runway releasing Praxis-1 under the Apache 2.0 license?
By open-sourcing Praxis-1's weights and source code under Apache 2.0, Runway allows anyone, including competitors, to build commercial products on top of the model without paying royalties. This democratizes access to robot control technology and provides developers and robotics founders with a cheaper starting point than training a world model from scratch.
How does Praxis-1 reduce the need for expensive robot-specific datasets?
Praxis-1 builds on a video model trained on ordinary footage rather than requiring large amounts of robot-specific data collection. By fine-tuning this general video understanding with some robot data and human action videos, the model can adapt to new robot systems without needing the traditionally expensive and time-consuming process of collecting comprehensive robot-specific datasets.
What was Runway's original business focus before developing Praxis-1?
Runway built its reputation on text-to-video generation tools that allow marketing teams and creators to generate commercial-quality videos without hiring camera crews. The company is now leveraging this same video generation and prediction expertise to enter the robotics space with Praxis-1.
Further Reading
- Praxis-1: From Video Pretraining to Robot Control - Runway Research
- Introducing Praxis-1: Runway's Open-Weight World Action Model - Runway Research
- Praxis-1 Is Now Available - Runway
- Praxis-1: Open-Source Video-Action Policy for Robot Arms - GitHub
- Runway Releases Praxis-1 Robot-Control Model With Open Weights and Commercial-Use Rights - Superpower Daily
