Nvidia unveils GR00T 2 to train robots using human video data
The new world action model allows robots to learn complex tasks from human demonstrations without needing specific, step-by-step robot training.
The short version
- GR00T 2 is a world action model that predicts robot movements and environmental changes based on video and instructions.
- The model can be applied to various forms, including robot arms, five-finger hands, and humanoid robots.
- The system achieved a 95 percent success rate on tasks when provided with five demonstrations each.
- Nvidia will release the model weights and specific tuning methods for different robots in December.
- Industrial partners like Foxconn and Siemens are already testing the technology for manufacturing and automation.
Why it matters
This technology reduces the massive amount of manual labor required to teach robots new skills. By letting robots learn from human video, companies can deploy versatile machines in factories and homes much faster.
Learning from human motion
Nvidia recently introduced GR00T 2, an artificial intelligence model designed to help robots perform precise physical tasks. The core innovation of this system is its ability to transfer skills shown by humans directly to robotic hardware. Instead of requiring engineers to program every single movement, the model learns from a massive dataset of human actions.
As reported by 디지털투데이, the system utilizes a 100,000-hour human demonstration dataset to train the AI. These videos are filmed from the operator's point of view, allowing the model to understand how humans interact with objects. This approach helps robots generalize behaviors that they might not have seen performed by a specific machine before.
The goal of this release is to lower the barrier for creating functional robots. By using human video data, Nvidia aims to reduce the burden of converting human movements into robot skills. This shift means developers do not have to perform as much task-by-task training for every new robot they deploy.
How the world action model works
GR00T 2 functions as a world action model, or WAM. A WAM is a type of AI that predicts a robot's next actions while simultaneously predicting the resulting changes in the surrounding environment. It processes recently captured video, specific task instructions, and the current condition of the robot to decide on the next move.
The architecture is versatile enough to be applied to many different types of hardware. It can power robot arms, five-finger robot hands, and even full humanoid bodies. Because it understands the relationship between actions and environment changes, it can handle complex interactions like grasping objects or fitting parts together.
For humanoid robots specifically, Nvidia created a lightweight version called GR00T 2 Turbo. This version runs on the Jetson Thor platform and uses a layered structure. A high-level reasoning model decides what the task is, while the Turbo model plans the motions. A separate control system then converts those plans into the actual movements of the robot's joints.
Handling errors and multi-step tasks
One of the most difficult challenges in robotics is handling unexpected errors during a sequence of actions. Nvidia addressed this by introducing functions for multi-step tasks and error recovery. An AI agent manages the overall work sequence and tracks the progress of the job while the GR00T 2 model handles the precise physical motions.
If a robot drops a part or hits a snag that stops its progress, the AI agent can recognize the failure. It then initiates a recovery procedure to get the robot back on track. This allows the robot to continue the task without human intervention, which is essential for complex industrial workflows.
The system is designed to be robust enough for real-world applications where things rarely go perfectly the first time. By combining high-level reasoning with low-level motion control, the robots can navigate through a series of steps. This combination makes it possible for machines to handle more than just simple, repetitive movements.
Performance and industrial testing
Nvidia's internal evaluations show significant progress in how quickly these robots can learn. In one test, a target robot performed 40 out of 50 tasks at least once without any specific demonstrations for that robot. This resulted in an average success rate of 40 percent for the baseline model.
The success rate improved dramatically as more demonstrations were added. The rate rose to 71 percent when the system was given one additional demonstration per task. When provided with five demonstrations per task, the success rate reached 95 percent, showing how effectively the model learns from human examples.
Several companies are already testing the technology in industrial settings. Foxconn is currently using GR00T 2 to automate a server manufacturing plant, and Siemens is evaluating the model for various industrial uses. Additionally, the Taiwan robot company Nexuni is testing the model specifically for tasks involving laundry handling.
The roadmap for developers
Nvidia is following a specific rollout plan to ensure that developers can integrate these tools into their own projects. They are providing the data and tools first so that the community can begin exploring the capabilities of the human video dataset. This allows for early experimentation with the underlying logic of the world action model.
The company plans to release the model weights and specific training and tuning methods in December. These weights are the learned parameters of the AI that allow it to function. Once released, developers will be able to apply the model to their own specific robots and customize it for unique tasks.
By providing these resources, Nvidia hopes to create a standard for how robots learn from human data. The ultimate goal is to make it easier for companies to deploy robots that can perform complex, multi-step jobs. This could lead to faster automation in manufacturing, logistics, and other sectors that rely on physical labor.
What to watch
- The release of model weights and tuning methods in December.
- How Foxconn and Siemens report the success of GR00T 2 in their specific manufacturing plants.
- Whether the 95 percent success rate holds up in unpredictable, non-laboratory environments.
Try it
Look up Nvidia's Jetson Thor platform to see what hardware currently powers their humanoid robot models.
Sources and links
- Reporting디지털투데이: the article we sourced
First reported by 디지털투데이. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.
Liked it? Say why
Add a line for other readers.
No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.