Nvidia boosts Nemotron 3 Ultra coding scores using 20,000 training examples
By using a new reinforcement learning method and a specialized coding agent harness, Nvidia significantly improved its model's ability to handle complex programming tasks.

The short version
- Nvidia used the Droid coding-agent harness to capture reproducible failures in instruction following and tool calling.
- The company employed PivotRL, a reinforcement learning method that trains models on critical decisions without full multi-turn rollouts.
- Nemotron 3 Ultra's Terminal-Bench 2.0 score jumped from 38.5% to 49% after the training.
- Nvidia used a multistage pipeline combining reinforcement learning with verifiable rewards, human feedback, and multi-teacher distillation.
Why it matters
This shows how specific, high-quality data can fix logical errors in AI agents without needing massive amounts of new compute. It provides a blueprint for making models more reliable at executing multi-step technical tasks.
Identifying specific AI failures
Nvidia recently detailed how it improved its Nemotron 3 Ultra model, a 550-billion-parameter open model released in June. To do this, the company worked with Factory to use a coding-agent harness called Droid. This setup allowed them to identify and capture reproducible failures in how the AI follows instructions, calls tools, and recovers from errors.
The Droid harness functions by relying on Nemotron to maintain a structured plan while it executes a task. During testing, Factory found that the model struggled with argument formatting and tool calls. In some cases, the model would continue working even after encountering an error without attempting to recover.
To isolate these issues, Factory evaluated the model against proprietary datasets like Legacy Bench. This helped the team identify the specific decisions behind unsuccessful task executions. Jian Zhang, who leads Nemotron post-training research at Nvidia, said the team analyzed these traces to improve planning and error recovery.
The PivotRL training method
To fix these mistakes, Nvidia turned to a reinforcement learning method called PivotRL. This method was introduced in March and focuses on training models on critical decisions within existing supervised fine-tuning trajectories. It avoids the high expense of full multi-turn rollouts, which are sequences of many back-and-forth interactions.
The reward mechanism in PivotRL is designed to evaluate whether an action achieves the intended final outcome. It does not simply check if the model's response matches a reference example. On agentic coding tasks, PivotRL achieved accuracy levels similar to end-to-end reinforcement learning but used only one-quarter as many rollout turns.
The method also showed significant gains across different domains. PivotRL outperformed standard supervised fine-tuning by an average of 4.17% across four agentic domains. It also showed a 10.04% improvement on out-of-domain non-agentic tasks, proving its versatility beyond just coding.
Data generation and distillation
Nvidia generated about 20,000 rows of post-training data for Nemotron 3 Ultra using the Droid harness. To prevent the model from over-fitting to a specific harness, the company varied Droid’s configuration and included data from several different agent environments. This variety ensures the model remains flexible across different platforms.
The training pipeline is a multistage process that combines reinforcement learning with verifiable rewards and human feedback. It also uses multi-teacher on-policy distillation. This process transfers capabilities from teacher models, which were trained across various technical domains, into the student model.
The final training configurations for terminal-use included several environments, such as OpenCode, Codex, and an environment called agent006. These were combined with the specific Droid-specific PivotRL data to refine the model's performance. Nvidia has published these reinforcement learning blends and instructions for reproducing the pipeline on GB200 NVL72 hardware.
Measurable gains in coding
The results of these improvements were significant on standardized benchmarks. Nemotron 3 Ultra’s score on Terminal-Bench 2.0 increased from 38.5% to 49%. This benchmark measures how well models can perform tasks in a terminal environment, which is a common interface for developers to run commands.
The success of this approach extended to other models as well. A preliminary checkpoint of Nemotron 3.5 Super VL saw its score improve from 52.11% to 60.4% on Terminal-Bench 2.1. These jumps suggest that targeted post-training can effectively bridge the gap between general knowledge and specific task execution.
While these results are promising, some questions remain about the specific contributions of the harness. Factory has not published an ablation study to isolate how much of the gain came from the Droid harness specifically. They also have not disclosed the before-and-after error rates for individual tool-use failures.
Alternative approaches in the field
Other companies are exploring different ways to optimize models for specific tasks. For example, Hugging Face’s Harness Hub team used reinforcement learning to train separate LoRA adapters. These are small, modular layers added to a model that change its behavior without modifying the base model weights or the harness itself.
Hugging Face reported that a version of Qwen 3.8 27B optimized for the Pi environment saw its Terminal-Bench 2.1 score rise from 39.5% to 55.8%. This optimization also reduced the number of tool calls per task by 55% to 62%. They also developed a rule-based proxy called safeagents to prevent reward hacking during the training process.
Currently, there is no clear consensus on which method is superior. Neither Nvidia nor Hugging Face has published evaluations that quantify the performance and computational costs of harness-specific adaptation versus multi-harness post-training. The industry continues to test whether it is better to train a model to be a generalist or to create specialized adapters for different tools.
What to watch
- Whether multi-harness post-training becomes the standard for production-scale agentic models.
- Future results from the Nemotron 3.5 Super VL benchmarks as more data is released.
Try it
Check out the Nemotron 3 Ultra model on Nvidia's official documentation to see the published reinforcement learning blends.
Sources and links
Read the original at thenewstack.io
First reported by thenewstack.io. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.
Liked it? Say why
Add a line for other readers.
No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.