OpenAI moves robotics researchers into its core AI team

OpenAI has reorganized its standalone robotics group. The company confirmed the change to Fortune on November 13. Robotics researchers moved into core research under Mark Chen, while applied robotics and hardware moved into the Applied organization led by Greg Brockman. The company said its robotics work would continue.

OpenAI told Fortune that general-purpose models such as GPT-6 Astra had shown they could handle some robotics tasks. That puts a model designed for work across text, code and images alongside purpose-built tools in robotics research. The published experiments show what such a model can do with camera views and a way to issue commands, and how long the system waits for the next decision.

From engineering drawings to motion commands

OpenAI’s November 3 engineering update described how its hardware and infrastructure teams already used Codex and ChatGPT. They could edit computer-aided designs, prepare manufacturing drawings and set up simulations. In one collaboration described by OpenAI, the input was a three-dimensional scan of an existing part. The result was an editable design file. In robotics control experiments, the result of a model call can instead be the next action a device takes.

Nvidia’s robotics researchers compared several ways to give GPT-6 Astra that responsibility. In one configuration the model chose the motion directly. In another it worked inside a persistent Codex session, with previous observations and decisions available. A third gave the model a monitoring role and left motion generation to a robotics model. Each arrangement used Astra, with different responsibilities assigned to it.

More successful trials came with a higher bill

Nvidia’s Astranomy study tested 120 simulated tasks involving picking up and placing objects. The direct configuration succeeded on 66.3 percent of trials. The configuration that used Codex reached 74.8 percent. Nvidia’s comparison lists Lakeside, the strongest robotics model shown from the benchmark leaderboard, at 51.0 percent. Each figure is a result on the RoboLab-120 simulation benchmark.

Applying published token rates to successful runs, Nvidia estimated an average of $1.96 per run for direct model calls and $36.40 for the Codex configuration. A model call in successful runs of the direct setup averaged 18.56 seconds. Nvidia counted the time required to execute the motions separately.

The simulator paused while the model decided what to do. If an object moves during that wait, a physical robot has to deal with a scene that may have changed. The study did not measure that response. The team ran five episodes per task and cautioned that the limited number of repeats made it difficult to establish whether smaller differences between configurations were statistically significant.

A separate preprint, JustRUA, posted November 11, tested changes to the software around Astra on a different simulator. Its authors evaluated 42 two-arm manipulation tasks in RoboDojo. They added earlier observations to the context available to the model and tested shorter bursts of motion, which gave it more opportunities to see where the robot had ended up. The researchers also tested giving it depth estimates derived from color images. In one comparison, adding that information reduced the number of times the gripper closed on empty space.

In that benchmark, the authors reported a score of 29.7 for the original configuration and 50.3 for their final configuration. Those were points on the study’s scoring scale. Nvidia’s percentage measured task success on a different test. JustRUA’s authors documented failures in their strongest configuration, too, including objects knocked out of position and items that slipped out of a gripper.