Naver Cloud tunes Mamba-2 kernels for NVIDIA B200, speeding up training 1.12x
3-line summary
- Naver Cloud published results from re-optimizing Mamba-2 compute kernels for NVIDIA's B200 GPU.
- The slowest kernel got up to 3.18 times faster and a single Mamba-2 layer about 1.33 times faster, for an overall pretraining speedup of about 1.12 times.
- The key was not a new algorithm but retuning settings left over from the A100 era to suit the B200.
On October 7, in the 59th installment of its CLOVA tech series, Naver Cloud shared results from optimizing Mamba-2 kernels for NVIDIA Blackwell (B200) GPUs. Kernels are the programs that actually carry out individual operations on a GPU, and the starting point was that software has to evolve with each new GPU generation to unlock its performance.
Mamba-2 handles long contexts with less compute and memory, and leading models such as NVIDIA Nemotron, Alibaba Qwen and Moonshot AI Kimi mix it with conventional self-attention in hybrid architectures. But the widely used open-source kernel package, mamba-ssm, couldn't use B200's new hardware features and kept A100-era settings that turned into bottlenecks.
The team rebuilt the kernels with Helion, which runs on top of Triton and automatically searches for the best settings. Fixing the top 10 kernels, which took 82% of total compute time, made them 1.43 times faster on average and the slowest one up to 3.18 times faster. The trick was the number of concurrent memory requests: B200 has about four times A100's memory bandwidth (2 TB/s to 8 TB/s) but not proportionally lower latency, so replacing a setting the old rule fixed at 74 with an autotuned 2,601 let the kernel actually use the wider bandwidth.
They also tested it in real training. Dropped into Nemotron-3 Nano 30B pretraining as a one-to-one replacement for the original package, the kernels made each training step about 1.12 times faster, and the loss converged stably over hundreds of thousands of steps. Naver Cloud plans to extend the work to larger models and multi-node distributed training.
Why it matters
The same GPUs can cost very different amounts to train on depending on how well the software fits them. A 1.12x gain sounds small, but at large scale it cuts real cluster time and cost — and it shows a Korean company building its own accelerator-optimization expertise.
Who it's for
- Developers
- If a new GPU underperforms, check whether your kernels' parallelism settings still assume the previous generation — here one setting made a more-than-threefold difference.
Sources
- [클로바 시선 #59] 최신 GPU의 잠재력을 깨우다: B200을 위한 Mamba-2 커널 최적화 — NAVER Cloud
