Nvidia releases Flora to cut robot video labeling costs by 125x

The challenge of robot training

Training large AI models for robotics requires a massive amount of video data paired with precise text descriptions. To help a robot learn through trial and error, researchers need three specific types of information for every video clip. These include descriptions of what actions occur and when, evaluations of how well a robot progresses step-by-step, and final judgments on whether a task was successful.

Most existing tools only provide one of these three types of data at a time. While powerful large language models like GPT-6 Astra can generate multiple types of information, they are often too expensive to use for the sheer volume of video required. This creates a bottleneck for developers who need to scale their training sets without breaking their budgets.

Nvidia's research lab, GEAR, developed Flora to solve this specific problem. The system is designed to provide all three types of annotations simultaneously. By automating this process, Nvidia aims to make it easier for researchers to create the high-quality datasets needed to train robots for complex physical tasks.

How Flora works technically

Flora is not a single model but a system composed of three different fine-tuned Qwen3.5 models. Each model has a specialized role in the data processing pipeline. A 9B-parameter model is used to break down videos into specific action segments. A 27B-parameter model then describes these segments while considering the events that happened before and after them.

The third component is called Flora-Rewarder, which is another 27B-parameter model. Depending on the prompt, this model can perform three different roles. It can rate the success of a robot's episode on a scale of 1 to 5, classify every half-second step as normal, suboptimal, or an error, and assign scores based on self-defined categories.

The system is already being used to train Nvidia's GR00T models. It is capable of annotating up to 200,000 hours of video per day on a cluster of 512 GB300-GPUs. By making these models open, Nvidia allows the broader research community to access these tools for their own robotic projects.

Cost and performance metrics

The most striking claim from Nvidia is the cost difference between Flora and competing models. According to research published by The Decoder, Flora can label an hour of robot video for about 72 cents. In contrast, GPT-6 Astra costs 88.50 dollars for the same task. This represents a massive reduction in cost, though Nvidia notes that the Flora price is a projection based on an assumed chip rental price of 10 dollars per hour.

In head-to-head tests, Flora showed specific strengths in action recognition. For example, in a video featuring many short movements, Flora identified 34 out of 37 individual actions, while Astra only caught four. However, the results for general descriptions are mixed. While Flora excels at identifying action boundaries, it sometimes lags behind models like Gemini 3.8 Flash in overall description accuracy.

One notable area of success for Flora is error detection. On the RoboRewardBench, Flora had 27.8 percent fewer errors than Astra. It was particularly effective at finding errors within a single episode, identifying 53.3 percent of error steps where other models failed to catch any. This capability is vital for teaching robots to recognize when something goes wrong during a task.

Strategic goals for Nvidia

By releasing Flora as an open tool, Nvidia is positioning itself as a primary provider for the robotics ecosystem. This move allows researchers to scale their own datasets and test methods for transferring human demonstrations to robots. It serves as a practical alternative to commercial services like Scale AI's Physical AI Data Engine.

There is also a clear business logic behind this open-source strategy. Nvidia invests billions in open AI models to keep developers within its own hardware ecosystem. By providing the tools for data processing, training, and simulation, Nvidia aims to capture the entire value chain of the robotics industry.

This release follows a broader strategy to address the shortage of robotics training data. Earlier this year, Nvidia introduced a data pipeline that automatically prepares, multiplies, and evaluates training material. Flora complements this by adding detailed labels and reward signals, helping to bridge the gap between digital simulations and real-world robot performance.

Limitations and next steps

Despite its strengths, Flora is not perfect and has some notable limitations. The researchers admitted that because GPT-6 Astra was used to evaluate its own performance in some tests, the results might be biased. Additionally, Flora can sometimes over-segment long actions into too many small steps, and it occasionally makes mistakes in its descriptions, such as describing a folded napkin instead of a wiped plate.

The model also has a false positive rate, marking about 11 percent of normal steps as errors. While it is highly effective at finding errors that other models miss, it still requires human oversight to ensure the highest quality of training data. Developers will need to balance the speed and low cost of Flora with the need for accuracy in critical tasks.

Looking ahead, the success of Flora will depend on how quickly it can be integrated into mainstream robotics workflows. As companies look for ways to move robots out of factories and into homes, the ability to cheaply label diverse human behaviors will be essential. Researchers will likely focus on refining Flora's description accuracy to match the high standards set by other large language models.

Try it today

You can find the Flora models available on Hugging Face to see how they perform on your own video datasets.

Watch

How will the false positive rate of Flora affect the speed of robot learning in complex environments?

Will other hardware providers release similar open-source data engines to compete with Nvidia's ecosystem?