NVIDIA GR00T 2 averages 95% on selected tasks after five demos each

NVIDIA introduced its GR00T 2 robot learning model at the Conference on Robot Learning in Austin on November 9. In a test on Dexter, a robot with two Sharpa hands, researchers recorded five demonstrations of each task by remotely operating Dexter, then used that data to adapt the model. It reported an average success rate of 95% across 50 selected tasks.

The test addresses how much new training data a robot needs to perform a skill that a person or another machine has shown. Dexter’s own pretraining data contained none of the 50 tasks. GR00T 2 had seen them in human video, recordings from a robot with a different gripper, or both.

How much the demonstrations changed the result

NVIDIA compared the same tasks under three conditions. The pretrained model received only a task instruction in the first condition. In the other two, researchers adapted it with either one or five demonstrations recorded on Dexter. With the instruction alone, it succeeded in 40% of trials. It completed 40 of the 50 tasks at least once, a separate count that includes tasks the robot sometimes failed.

One demonstration brought the average success rate to 71%. Five demonstrations took less than three minutes of robot data per task, and each fine-tuning run took about half an hour on one RTX 5090. Under the five-demonstration condition, the robot succeeded every time on 37 of the 50 tasks. Unloading a dish rack and unstacking Rubik’s cubes were among the tasks it always completed without new demonstrations. Emptying a bag and unfolding a tripod went from zero successes with the instruction alone to success in every trial after five demonstrations.

The technical report describes how the researchers chose those tasks. They drew the set from tasks on which the model reached at least 80% success after training on five demonstrations. The authors say that choice contributed to the high average in the five-demonstration condition. The 95% figure describes the result on those tasks with Dexter’s particular hardware.

Learning from a person’s view of a task

GR00T 2 uses recent camera footage, a task instruction and the robot’s current state to predict both a future view and a sequence of movements. It executes a sequence, receives another observation and predicts the next movements. Human recordings can contribute to training because the model also learns how objects and scenes change over time.

NVIDIA’s pretraining mix included roughly 400,000 hours of first-person human video, about 10,000 hours of handheld-gripper demonstrations and 23,500 hours of people operating robots remotely. The mixture brings together several ways of observing a skill, including data from different grippers and hands.

In a separate transfer experiment, people demonstrated how to assemble a toy dump truck with a handheld gripper. NVIDIA then tested a YAM arm with a different gripper. The company reported 60% success on the full assembly without using a demonstration of that task recorded on the YAM arm. NVIDIA describes the full assembly as a task that can take up to four minutes.

Part of the training data is public

NVIDIA made 100,000 hours from its Human-1M collection available through Hugging Face. The data card describes first-person footage paired with task descriptions, hand motion and head-camera position and orientation. An explorer groups examples by type of work and setting. One sample shows a worker packing boxes and applying shipping labels. Synchronized head and wrist views show the bench and the movements close to the worker’s hands. The public subset carries a Creative Commons Attribution 4.0 license.

NVIDIA says Foxconn is applying GR00T 2 to server manufacturing automation and Siemens is evaluating industrial uses. The research page lists December for the code release. NVIDIA robotics director Jim Fan also said the weights and training recipes for transferring skills to different robots are planned for December.