DEEP Robotics unveils five research advances for embodied AI

Grounding robots in physics

DEEP Robotics recently introduced DR-WM-1.0, a world model designed to help robots predict the physical consequences of their actions. While many current models can generate visually convincing videos, they often fail to account for actual physics, such as a robot slipping on ice. This new model addresses that gap by incorporating specific robot-terrain contact dynamics.

The system combines reconstructed scene geometry with physical priors for factors like friction, roughness, and stiffness. It was calibrated using more than 100,000 real-robot clips and 2.58 million image-mask pairs. By using a dual-ControlNet architecture, the model conditions video generation on both scene geometry and the camera motion of a simulated robot.

This allows the robot to perform a form of 'physical rehearsal' before moving. For example, a robot inspecting a substation after a storm could simulate a slippery shortcut versus a longer route. It can then weigh the risk of unstable motion against the efficiency of the path before actually committing to the movement.

Learning from human experience

To help robots navigate streets and campuses, DEEP Robotics developed DR-VLA-2.0. This model aims to teach robots how to follow instructions and avoid obstacles by learning from human behavior. Because collecting robot-specific demonstrations is expensive, the team used 10,000 hours of curated egocentric web videos from humans to help the robot understand scene layouts.

The model uses a Mixture-of-Transformers framework to combine this human knowledge with 2,000 teleoperated robot navigation clips. This allows the robot to focus on what is present in a scene and how different elements relate to one another. The goal is to enable socially compliant behaviors, such as yielding to pedestrians or bypassing obstacles in residential communities.

Testing on a specific split showed that DR-VLA-2.0 improved the Quadruped Navigation Score by approximately 5% compared to the best competing result. It also reduced the open-loop final heading error by 14.6% relative to the best competitor. These metrics combine safety, instruction compliance, comfort, and progress to measure how well a robot moves in shared spaces.

Language-steerable manipulation

Translating human language into precise physical motion is another hurdle addressed by the Atomic Motion Coordinate (AMC) framework. This system defines 13 interpretable motion atoms for each robot arm, including translations and rotations along three axes as well as a hold action. By first training without visual inputs, the model learns to align language directly with geometric motion.

The framework then incorporates visual context and force feedback to adjust actions in real-time. For instance, in tasks like wiping a vase or plugging in a connector, the robot must adapt to physical resistance. The evaluations show that adding force feedback raises task progress to 78.5% for a drawer-and-plug task and 75.2% for vase wiping.

The research demonstrates significant improvements in how well a robot can be steered by language compared to baselines like fine-tuned π0.5. AMC achieved opposite-atom separation rates of 92.5% for single-atom prompts and 83.1% for two-atom prompts. These results suggest that robots can become more reliable at performing complex, contact-rich tasks in varied layouts.

Optimizing diverse policies

As robot models become more complex, training them requires methods that work across different types of action representations. DEEP Robotics and Zhejiang University developed DR-APO to separate policy improvement from the underlying representation. This means the same optimization principle can be used for Gaussian, diffusion, and flow-matching policies.

DR-APO uses an advantage-guided target improvement mechanism to construct better action targets based on local gradients. This approach avoids the need for explicit policy-likelihood evaluation during updates. It is designed to prevent local improvement signals from being diluted during the multi-step generation process used by many modern models.

The framework was tested across five MuJoCo-v5 continuous-control tasks, where it achieved the best result in nine of ten diffusion-policy comparisons. In real-world navigation tests on a wheel-legged robot, DR-APO raised the success rate from 25% to 65%. This provides a versatile foundation for training robots across a wide variety of different control architectures.

Memory and real-world data

Reliable long-term operation requires robots to remember things they can no longer see. DR-Agent addresses this by using a predictive 4D belief system that constructs a time-indexed history of 3D objects. If an object is moved in a warehouse or a hotel room, the robot can estimate its likely new location based on temporal patterns and previous observations.

This predictive memory helps the robot revise its plans as the environment changes. The researchers tested this using the EvoWorld-Bench, which contains 54 simulated scenes and 803,680 task instances. The system showed clear gains in navigation success, particularly when the robot could learn patterns of how objects move over time.

To support all these advances, DEEP Robotics also released DR-Benchmark, which includes the LynxScenes dataset. This dataset covers 833.6 hours of real-world operation and includes 3 million annotated frames. It provides a unified way to evaluate perception, navigation, and future scene prediction using consistent real-world data across multiple tasks.