Study finds Nvidia Blackwell traces in the starting weights of DeepSeek and Kimi

Independent researcher Will DePue reported on November 12 that public files for six AI models, including DeepSeek V4 and Kimi K3, contained numerical patterns consistent with Nvidia’s Blackwell GPUs. The study compared values left over from when developers first set up the models. The findings give researchers a way to investigate one step in model development by examining downloadable files. The report leaves the chips’ location, what hardware ran the rest of training, and whether any export rules were breached unresolved.

DePue’s group, Unemployment Labs, also released code and a selection of weights for checking four of the six Blackwell results and a comparison model, DeepSeek V3. The weights are the numbers a model uses to produce its answers. The public files let other researchers check parts of the analysis without rebuilding the original training runs or relying on a company to disclose its computers.

The traces left in model files

Before developers train a model, they fill its weights with starting values. Training alters many of them. Entries for some unused text tokens can retain their initial values, which makes those rows candidates for comparison. DePue’s analysis looks at the numbers stored in the model files. It does not infer the chips from the text a chatbot happens to generate.

PyTorch lets developers specify a seed, the starting input for its random-number generator. The GPU setup examined in the study shares that work among parallel workers. A different layout can put the generated numbers in different places, even with the same seed. The report describes how the researcher looked for a layout in one sample and checked it against other values. That second comparison tests whether a match carries across the file.

The report links DeepSeek V4 Flash and MiniMax M3 to a layout associated with the B200 and B300. DeepSeek V4 Pro, Moonshot AI’s Kimi K3 and K3.1 Pro, and Ant Group’s Ling 3.1 Flash match a layout associated with the GB200 and GB300. Older DeepSeek V3 had a layout consistent with the Hopper generation. Several Hopper chips share that layout, leaving the specific chip uncertain from the pattern alone.

In DePue’s hardware tests, a B300 reproduced 445 of the 469 selected V4 Flash rows bit for bit. An H100 reproduced preserved rows from DeepSeek V3 and gave no match on the selected V4 Flash rows. His accompanying repository also records a comparison for 13 Kimi K3 rows on a GB200. The physical tests use the standard PyTorch functions and compare their output with the values from the model files.

What others reproduced and what remains unknown

Chris von Csefalvay published a separate implementation that recovered the reported layouts for DeepSeek V3, DeepSeek V4 Flash and Kimi K3. His results show high correlations on rows held back when choosing a candidate, and much weaker correlations with an incorrect seed. That comparison ran on a regular processor. His repository says the scripts for bitwise tests on physical GPUs remain unrun.

DePue also found that a specially configured AMD MI300X could produce the same layout. He made a finer comparison for V4 Flash using locations where Nvidia and AMD generate slightly different values. The published weights supported the Nvidia result. The report says the same test did not settle which vendor’s chips initialized the other five models.

Nvidia’s filing for the quarter ended in July lists the B200 and GB200 NVL72 among products that require a US license for export to China. A weight pattern contains no address, ownership record or shipping document. The method also depends on accessible weights that preserve a recognizable pattern. DePue reported no match for DeepSeek’s later V4.1 Flash, leaving the hardware that initialized that model unclassified.