Nvidia releases GR00T 2 to train robots using human videos
The new world action model allows robots to learn complex tasks from human footage, potentially cutting down the time needed to train machines for factory work.
The short version
- GR00T 2 learns from first-person human video and tens of thousands of hours of robot data.
- The model achieved a 95% success rate on tasks using only five demonstrations and 30 minutes of fine-tuning.
- Nvidia is releasing a 100,000-hour human video dataset for both commercial and noncommercial use.
- The system is being tested by companies like Foxconn and Siemens for manufacturing and industrial automation.
Why it matters
This technology could significantly lower the barrier for businesses to automate new tasks because it requires less manual "teaching" of the robot. It moves the needle toward robots that can learn by watching humans rather than requiring constant, precise programming for every new job.
Learning from human observation
Nvidia recently introduced GR00T 2, a world action model designed to train robots more efficiently. Instead of requiring a human to manually guide a robot through every single movement, the system can learn by watching videos of people performing the same actions. This shift could make it much faster to deploy robots in environments where tasks change frequently.
The model was unveiled at the Conference on Robot Learning in Austin, Texas. It is built on the Nvidia Cosmos framework, which allows the AI to predict future video frames and robot movements simultaneously. By doing this, the system learns how specific physical movements actually change the surrounding environment.
To train the model, Nvidia used a massive amount of data, including 402,730 hours of first-person human video. They also included 10,184 hours of handheld-gripper demonstrations and 23,500 hours of robot teleoperation data. This diverse dataset helps the AI understand the relationship between human intent and physical execution.
Performance in manipulation tests
In testing involving 50 different manipulation tasks, the results showed significant progress in how quickly robots can pick up new skills. The model completed 40 of those 50 tasks at least once without any specific demonstrations on the target robot. This suggests the AI can generalize knowledge from its broad training to specific, new scenarios.
When robots were given a few examples, the success rates climbed even higher. The system reached a 71% average success rate with just one demonstration per task. With five demonstrations, the success rate jumped to 95%, which required less than three minutes of robot data and about 30 minutes of fine-tuning on an Nvidia RTX 5090.
However, the technical paper notes that the 50-task evaluation set was pre-selected from tasks that already reached at least 80% success after five demonstrations. This means the current results do not prove that the model can perform equally well on every possible arbitrary task a business might encounter.
Simulated physics and precision
Alongside GR00T 2, Nvidia introduced Isaac Lab 3.0, which is an open-source framework for training robots in digital simulations. This tool integrates with the Newton physics engine to help researchers model very complex interactions. It can simulate how robots handle difficult materials like cables and cloth, or how they interact with human-like hands.
Using the Newton engine provided a significant boost in training speed. Nvidia reported that Isaac Lab 3.0 achieved up to 2.5 times the training throughput compared to the previous version using the PhysX engine. This speed increase was measured in a benchmark focused on vision-based dexterous manipulation, where robots must use precise finger movements.
Another project mentioned, AlphaNIST, used these simulation techniques to train a single policy for 20 different assembly tasks involving millimeter-scale components. It achieved a 98.2% success rate in simulation while using approximately one thirty-ninth as much GPU time as separate, task-specific policies. These results demonstrate how simulation can streamline the heavy lifting of AI training before moving to physical hardware.
Real-world industrial adoption
Several major companies are already beginning to test GR00T 2 in practical industrial settings. Foxconn is currently applying the model to automate server manufacturing, while Siemens is evaluating it for various industrial applications. Additionally, a company called Nexuni is testing the technology to handle robotic laundry tasks.
For manufacturers, the primary advantage is the reduction in time spent collecting specific robot demonstrations. This could make it much easier for a factory to adapt its automation to new products or changing components. If a robot can learn a new job by watching a video, the transition from a human-led process to a robotic one becomes much smoother.
While the technology is promising, the research does not yet show how much a business will actually save once they factor in the costs of integration and ongoing supervision. The 30-minute fine-tuning time mentioned by Nvidia only covers one adaptation step. Businesses will still need to account for the full process of preparing a robot for a production line, including testing for reliability and hardware compatibility.
What to watch for next
As these models move from research labs to factory floors, the focus will shift toward operational reliability. Companies will need to measure how often human workers must intervene during a shift and how many retries a robot needs to complete a single task. Success in a lab does not always translate to high throughput in a high-speed production environment.
IT and operations teams should also look closely at the total computing costs and hardware requirements. While the model is powerful, it requires specific hardware capacity and compatibility with existing cameras and control software. Determining who owns the responsibility for updating the model and monitoring its behavior will be a key hurdle for large-scale deployment.
A good starting point for any business is to pilot the technology on one repeatable task with a clear baseline. By comparing the training effort and cycle times against existing human-led processes, companies can determine the true value of the model before expanding to more complex and delicate work. Nvidia plans to make the code for GR00T 2 available in December.
What to watch
- How much cost savings will manufacturers see after accounting for full integration?
- How the model performs on tasks that were not pre-selected for high success rates.
- The release of the 100,000-hour Human-1M-100K dataset for commercial use.
Try it
Check out Nvidia's research page to see the upcoming code availability for GR00T 2 in December.
Sources and links
- ReportingTechRepublic: the article we sourced
First reported by TechRepublic. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.
Liked it? Say why
Add a line for other readers.
No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.