Oak Ridge opens Lux supercomputer with 4,032 AMD GPUs to researchers
Edited and reviewed by AI Ping Editorial Team
3-line summary
- Oak Ridge National Laboratory opened Lux to scientific users for research that combines AI training and simulation.
- The Department of Energy’s Genesis Mission gets about half of the 3.5 million annual node-hours. AMD makes part of its share available to paying partners.
- Lux debuted at No. 14 in November’s TOP500 after scoring 220.79 petaflops on the 64-bit Linpack benchmark.
Oak Ridge National Laboratory opened its Lux supercomputer to scientific users on November 16. The US Department of Energy lab added a system that can run AI model training and high-performance simulations for the same research project alongside its existing Frontier machine. Lux made its TOP500 debut that day at No. 14 in the November supercomputer rankings.
Oak Ridge built Lux with AMD and Hewlett Packard Enterprise. The lab’s user guide lists 4,032 AMD Instinct MI355X GPUs across 504 compute nodes. A node is one server in the larger system. Each Lux node has eight GPUs, each with 288 GB of high-bandwidth memory for holding models and data close to the processor during a calculation.
Where the 3.5 million node-hours go
Oak Ridge lists 3.5 million node-hours a year for Lux. Running one node for one hour uses one node-hour. Running ten nodes for two hours uses 20. A team can draw down its allocation quickly with a large parallel job or spread the same computing time over a series of smaller jobs.
About half of the annual capacity goes to the Department of Energy’s Genesis Mission. AMD receives the other half for development and testing and makes some of its time available to public and private partners on a paid basis. Oak Ridge says paid projects can start in any month and last from six months to five years. The minimum commitment is 175,000 node-hours per six months. Interested institutions discuss the application process with Oak Ridge. The facility’s Lux page gives a minimum in computing time and does not list a dollar rate per node-hour.
How simulation and AI fit into one project
Researchers might run a simulation, use its output to train a model, and then use that model to generate predictions. Those steps can put different demands on computing time and memory. Lux has separate partitions managed by Slurm and Kubernetes. Slurm lets researchers request resources and time for jobs in a queue. Kubernetes assigns resources to jobs packaged in containers, which bundle the software environment a job needs to run.
Oak Ridge says a Lux allocation includes access to its Orion shared file system. Groups with a Frontier allocation can write simulation output to Orion and use it as training data on Lux. A team using both systems can plan where it runs each stage and keep the intermediate dataset on the shared storage.
Oak Ridge listed a project to forecast streamflow, river temperatures and water availability as one of the Genesis Mission assignments for Lux. The project aims to support power generation in the Tennessee Valley with forecasts spanning weeks to years. Another assignment will develop AI models for materials research and manufacturing. The lab disclosed the project goals and the allocation of computing time. Its announcement gave no results on forecast accuracy or use in power plant operations.
What Lux measured at No. 14
The No. 14 ranking reflects Lux’s score of 220.79 petaflops on the High Performance Linpack benchmark. Linpack measures the rate of 64-bit floating-point calculations when a machine solves a large system of linear equations. TOP500 lists that measured score as Rmax. It puts Lux’s theoretical peak, or Rpeak, at 323.90 petaflops.
Frontier held No. 3 on the same November list. South Korea’s Hangang system at the Korea Institute of Science and Technology Information entered at No. 12. The rankings give those machines a common yardstick for one class of numerical work. Measuring training time or the accuracy of a scientific forecast calls for a test with the model and data that the research team actually uses.
Sources
- Lux debuts in the TOP500 and opens to scientific users — Oak Ridge Leadership Computing Facility
- AMD-powered Lux opens to scientific users at Oak Ridge — AMD
- Lux system overview and terms for partners — Oak Ridge Leadership Computing Facility
- Lux User Guide — Oak Ridge Leadership Computing Facility
- November 2026 TOP500 list — TOP500
- Lux system entry and measured Linpack performance — TOP500
- The Linpack Benchmark — TOP500
