NVIDIA delivered a clear sweep in MLPerf Coaching v6.0, the most recent version of industry-standard AI coaching benchmarks developed by the MLCommons consortium. NVIDIA achieved the quickest time to coach at scale, and in addition delivered the very best efficiency when normalized on a per-accelerator foundation on each benchmark. It was additionally the one platform to submit on each check.
MLCommons launched new pretraining benchmarks on this spherical designed to replicate the most recent developments in AI fashions, together with DeepSeek-V3, an enormous 671B-parameter Combination of Consultants (MoE) mannequin that additionally serves as the bottom for the favored DeepSeek-R1 reasoning mannequin, and GPT-OSS-20B, a small-but-capable MoE.
The NVIDIA platform was the one one to submit outcomes on each new workloads, with the NVIDIA GB300 NVL72 system setting the efficiency bar via optimized NVIDIA software program stacks and a design that connects 72 NVIDIA Blackwell Extremely GPUs and 36 NVIDIA Grace CPUs as one utilizing NVIDIA NVLink and NVIDIA NVLink Swap.
Unprecedented scale and throughput throughout the scale-out cloth
Coaching state-of-the-art fashions requires large-scale infrastructure and the power to effectively execute workloads throughout hundreds of interconnected processors. In a number of entries this spherical, NVIDIA cloud service supplier companions scaled as much as 8,192 Blackwell GPUs working in unison throughout various cloud information facilities. These submissions proved the real-world robustness of the Blackwell platform throughout manufacturing hyperscale information heart fleets, demonstrating robust scaling developments throughout these diversified cluster environments.
Extracting most effectivity from every coaching iteration at this magnitude requires transferring far past the attain of a single NVLink area, counting on scale-out networking platforms equivalent to NVIDIA Spectrum-X Ethernet and NVIDIA Quantum InfiniBand. Skilled parallelism inside MoE fashions generates low-entropy, bursty flows—a sample that sometimes reduces efficient bandwidth below static Equal-Price Multi-Path (ECMP) hashing as massive flows collide on shared hyperlinks.
To resolve this, Spectrum-X Ethernet’s Superior Adaptive Routing distributes site visitors packet-by-packet throughout all accessible paths in line with real-time hyperlink load, sustaining efficient bandwidth close to the material’s theoretical capability whereas the receiving ConnectX SuperNIC handles out-of-order supply.
Moreover, when a preferred professional attracts simultaneous site visitors from many senders, Spectrum-X Congestion Management makes use of real-time telemetry to detect the ensuing incast early and tempo senders earlier than buffers overflow. This balances tail latency so all-to-all communication stays hidden behind compute quite than surfacing on the primary execution path.
This mixture of cluster orchestration and community cloth effectivity enabled new time-to-train information throughout probably the most difficult benchmarks, as summarized beneath:
MLPerf Coaching v6.0 outcomes retrieved from www.mlcommons.org on June 16, 2026, from the next entries: 6.0-0005, 6.0-0102, 6.0-0001, 6.0-0015, 6.0-0102, 6.0-0101 and 6.0-0062. The MLPerf identify and brand are logos of MLCommons Affiliation in the US and different international locations. All rights reserved. Unauthorized use is strictly prohibited. See www.mlcommons.org for extra data.
The software program innovation engine
{Hardware} capabilities are solely pretty much as good because the software program driving them. To extract most efficiency for advanced MoE fashions like DeepSeek-V3, NVIDIA deployed a number of cutting-edge software program optimizations on this spherical of MLPerf Coaching:
1. Full-iteration CUDA graphs for token-dropless MoEs
Traditionally, token-dropless MoE architectures struggled to run absolutely inside CUDA graphs resulting from dynamic routing behaviors that compelled steady CPU-GPU synchronizations. For MLPerf Coaching 6.0, NVIDIA applied full-iteration CUDA graphs for the primary time for these MoEs. Two main hurdles had been addressed to be able to allow this. First, professional module operators, such because the quantizer, grouped GEMM, and token dispatcher, had been transitioned to a synchronization-free mode. On this configuration, enter shapes are derived instantly from GPU values, eradicating the need for host-side coordination. Second, system reminiscence was managed with out host involvement through paged stashing. This system permits fine-grained administration on pre-allocated GPU reminiscence, guaranteeing the method is absolutely suitable with CUDA graphs.
By rewriting crucial execution paths to remove all CPU-GPU sync touchpoints, the complete iteration workload was offloaded fully to the GPU. This eliminated the CPU from the crucial path and eradicated the overhead stemming from the variation of host execution, which might in any other case create cascading overhead delays when scaling to clusters of two,000+ GPUs.
2. CuTe DSL and kernel fusions
To realize the fusion of memory-bandwidth certain layers with grouped GEMM operations and the synchronization-free execution required by CUDA graphs, NVIDIA leveraged CuTe DSL for superior kernel fusions. This enabled builders to mix math and memory-handling operations instantly on the {hardware} layer, retaining information native to the registers and keep away from costly round-trips to world reminiscence. Moreover, help for dynamic tile scheduling hid unfused reads and writes behind GEMM operations, enabling an environment friendly overlap with communication kernels.
CuTe DSL additionally enabled the implementation of kernels that may eat form arguments instantly from GPU reminiscence which might be computed by one other GPU kernel beforehand. This means eliminated the necessity for CPU-GPU synchronization even for dynamic shapes that aren’t identified till runtime, fully eradicating the CPU from the crucial path for token-dropless MoEs. Along with the enablement of CUDA graphs, these superior fusions present greater than 8% end-to-end profit on Deepseek-v3 and a 93% end-to-end speedup on GPT-OSS.
3. MXFP8 consideration block
Historically, MoE coaching workloads have used 16-bit precision for consideration computation. This spherical, an MXFP8 consideration recipe was developed for improved efficiency with out impacting mannequin high quality. This supplied an end-to-end speedup for DeepSeekv3 benchmark whereas preserving the usual math required for consideration operations. This recipe retains the enter tensors of all batched-matrix-multiply operations within the consideration block in 8-bit precision, profiting from quicker FP8 math execution on the {hardware} in comparison with 16-bit floating level datapath. This kernel is offered in cuDNN via the Transformer Engine library.
4. Router and hybrid EP optimizations
The MoE router is used to dynamically assign tokens to specialised professional layers, making its efficiency an essential think about cluster-wide coaching bottlenecks. A number of elementwise kernels had been fused within the router, together with top-k and rating computations to reinforce efficiency. To maximise {hardware} utilization, these kernels had been transitioned from FP64 to FP32 math operations. This optimization delivered a kernel speedup of 5x. Moreover, a number of elementwise metadata processing kernels had been fused inside HybridEP, complemented by devoted efficiency tuning of the important thing permute/unpermute kernels. General, these optimizations yielded a efficiency acquire of 5% end-to-end.
5. 1F1B all-to-all overlap optimizations
A devoted 1F1B (One Ahead, One Backward) all-to-all (A2A) overlap scheme was beforehand launched into Megatron-Core to cover MoE communication behind computation on the batch stage. For this MLPerf spherical, the execution effectivity of this scheme has been considerably improved. Whereas 1F1B scheduling initially launched notable CPU overhead, capturing the complete iteration inside a CUDA Graph efficiently eradicated this host-side bottleneck. Efficiency was additional enhanced by prioritizing the communication stream, using dynamically scheduled CuTe DSL kernels, and enabling delayed weight gradient (wgrad) help for brand new cuteDSL GEMMs. Within the regular state, these changes achieved almost 100% A2A communication overlap, leading to an total 8% efficiency profit.
6. Minimizing imbalance between pipeline levels
As particular person computational kernels get quicker, underlying imbalances between pipeline parallel levels turn into extra pronounced. NVIDIA optimized the structure and stability of those pipeline parallel levels, minimizing structural idling (“bubble time”).
Pipeline imbalance is a significant bottleneck in pipeline parallelism (PP) effectivity. For DeepSeek-V3, the mannequin makes use of a hybrid layer setting with three dense layers on the entrance and Multi-Token Prediction (MTP) plus logits GEMM with crossentropy on the finish. To resolve this difficulty, Megatron-Core’s versatile pipeline structure help was leveraged to rigorously stability the levels, whereas MXFP8 precision was adopted for the logit projection GEMM to cut back its execution time on the crucial path. Utilizing MXFP8 for the logit projection GEMM didn’t influence numerical stability of the benchmark. These changes efficiently diminished pipeline imbalance to lower than 1%, translating to a 4% E2E efficiency financial savings.
Steady full-stack co-design: Sum of all of the components
Whereas standardized benchmarks seize point-in-time efficiency metrics, a significant driver of precise developer worth is the continual trajectory of software program optimization. Over the past three months, shut collaboration between {hardware} and software program engineering groups has unlocked important optimization milestones for NVIDIA platforms.
This fast tempo of innovation spans the complete NVIDIA software program stack. Reasonably than counting on optimizations in a single remoted layer, the above-mentioned improvements illustrate how parallel efficiency enhancements had been engineered throughout a number of foundational CUDA-X libraries, frameworks and APIs together with cuDNN, Transformer Engine, CuTe DSL, Megatron Core, and cuBLAS. Megatron Bridge serves because the central packaging layer that integrates these cross-stack enhancements, making them instantly accessible to builders in a unified ecosystem.
To exhibit this utilizing the most recent NVIDIA NeMo container 26.06 launch, the coaching efficiency of the NVIDIA Blackwell Extremely GB300 on DeepSeek-V3 improved 1.3x going from 1,298 TFLOPS/GPU to 1,648 TFLOPS/GPU (6,338 tokens/sec/GPU). This efficiency uplift in three quick months is the direct product of full-stack co-design, the systematic elimination of micro-bottlenecks throughout communication protocols, routing layers, and compute kernels all with out requiring modifications to the underlying silicon.
This steady optimization trajectory instantly elevates NVIDIA Goodput by squeezing out system overhead and maximizing the share of time GPUs spend doing helpful work. Consequently, infrastructure operators don’t simply get excessive theoretical peak capabilities, they get a mature platform that converts these uncooked FLOPS into steady, productive coaching progress. This allows present infrastructure deployments to seize instant coaching effectivity dividends because the software program ecosystem matures.
Platform comparability: Blackwell Extremely GB300 vs. GB200
Past software program good points, evaluating configurations inside the Blackwell household illustrates how delicate {hardware} changes complement full-stack optimization. As proven in Determine 2 beneath, Blackwell Extremely GB300 supplies a major coaching efficiency uplift over the baseline Blackwell GB200 throughout each dense foundational fashions and complicated MoE techniques.
MLPerf Coaching v5.1 and v6.0, closed division. Outcomes from entries: 6.0-0022, 6.0-0102, 6.0-0017, 6.0-0078, 5.1-0072, 6.0-0013, 5.1-0067, and 6.0-0031. The MLPerf identify and brand are registered and unregistered logos of MLCommons Affiliation in the US and different international locations. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for extra data.
This speedup stems from two main benefits: larger reminiscence capability and a bigger energy price range. Deep MoE architectures are extremely memory-bound throughout large-scale coaching.
The expanded reminiscence of the GB300 accommodates the added reminiscence overhead launched by full-iteration CUDA graphs with out requiring sub-optimal configurations or layer recomputation. Moreover, elevated reminiscence capability permits builders to make the most of smaller model-parallel communication teams. By retaining bigger parts of the mannequin native to the chip, the system spends much less time ready on cross-GPU communications, translating on to larger operational throughput.
Full-stack innovation and scale in MLPerf Coaching 6.0
The MLPerf Coaching 6.0 outcomes firmly set up NVIDIA’s full-stack method because the definitive commonplace for accelerating advanced generative AI workloads throughout the {industry}. By securing a clear sweep and successful each single benchmark on this spherical, the platform demonstrated unmatched execution pace in time-to-train metrics. Whether or not coaching ultra-dense foundational fashions or navigating the intricate token-routing mechanics of large MoE architectures, NVIDIA delivers unmatched efficiency throughout the board.
These benchmark successes are propelled ahead by a fast velocity of software program innovation, steady excessive co-design, and the maximized effectivity of NVIDIA’s Goodput. By engineering breakthroughs applied throughout Megatron Bridge, cuDNN, and the Transformer Engine, together with full-iteration CUDA graphs, CuTe DSL kernel fusions, and communication and pipeline optimizations, NVIDIA prospects frequently extract large efficiency good points instantly from the software program layer. This fast tempo of optimization permits builders to seize instant coaching effectivity dividends on their present infrastructure investments because the software program ecosystem matures.
Finally, the true metric of enterprise readiness is efficiency delivered at most deployment scale. The NVIDIA platform efficiently demonstrated robust scaling as much as 8,192 energetic GPUs working concurrently on production-ready cloud architectures. This confirmed functionality to orchestrate large coaching clusters ensures that enterprises can reliably compress commonplace multi-month coaching cycles right into a matter of minutes or hours, dramatically accelerating the time-to-market for the subsequent era of AI breakthroughs.
