---
description: NVIDIA Sets MLPerf Inference v6.0 Records with Blackwell Ultra Platform
title: NVIDIA Sets MLPerf Inference v6.0 Records with Blackwell Ultra Platform
image: https://www.storagereview.com/wp-content/uploads/2026/04/Storagereview-nvidia-end-to-end-graphic-mlperf-v6.jpg
---
[![Storage Review]()](https://storagereview.com/)
---
[Facebook](https://www.facebook.com/@storagereview) [X (Twitter)](https://twitter.twitter.com/storagereview) [Instagram](https://www.instagram.com/storagereview/) [LinkedIn](https://www.linkedin.com/company/storagereview-com) [YouTube](https://www.youtube.com/user/storagereview?sub_confirmation=1) [Email](mailto:info@storagereview.com) [Spotify](https://open.spotify.com/show/1y6VnznABhHeOSMOmbDTz0) [Reddit](https://www.reddit.com/r/StorageReview/) [Discord](https://discord.gg/TwMHb4azdC) [RSS Feed](https://www.storagereview.com/rss.xml) [TikTok](https://www.tiktok.com/@storagereview)
[Facebook](https://www.facebook.com/@storagereview) [X (Twitter)](https://twitter.com/storagereview) [Instagram](https://www.instagram.com/storagereview/) [LinkedIn](https://www.linkedin.com/company/storagereview-com) [YouTube](https://www.youtube.com/user/storagereview?sub_confirmation=1) [Email](mailto:info@storagereview.com) [Spotify](https://open.spotify.com/show/1y6VnznABhHeOSMOmbDTz0) [Reddit](https://www.reddit.com/r/StorageReview/) [Discord](https://discord.gg/TwMHb4azdC) [RSS Feed](https://www.storagereview.com/rss.xml) [TikTok](https://www.tiktok.com/@storagereview)
[](https://www.storagereview.com)
≡ Menu
- [Home](https://www.storagereview.com/)
- [Storage Reviews](https://www.storagereview.com/review)
- [End User Reviews](https://www.storagereview.com/consumer)
- [Enterprise Reviews](https://www.storagereview.com/enterprise)
- [Ubiquiti Reviews](https://www.storagereview.com/best/ubiquiti-reviews)
- [SR Merch](https://store.storagereview.com)
- [Leaderboards](https://www.storagereview.com/best)
- [Best Storage Arrays](https://www.storagereview.com/best/storage-arrays)
- [Best Enterprise SSDs](https://www.storagereview.com/best/enterprise-ssds)
- [Best Servers](https://www.storagereview.com/best/servers)
- [Best Enterprise GPUs](https://www.storagereview.com/best/enterprise-gpus)
- [Best Desktops for Local AI](https://www.storagereview.com/best/desktops-local-ai)
- [Best Laptops for Local AI](https://www.storagereview.com/best/laptops-local-ai)
- [Best Local LLM Tools](https://www.storagereview.com/best/local-llm-tools)
- [Agentic AI Hardware](https://www.storagereview.com/best/agentic-ai-hardware)
- [Best SSDs & Hard Drives](https://www.storagereview.com/best_drives)
- [Best Portable SSDs](https://www.storagereview.com/best/portable-ssds)
- [Best Business Laptops](https://www.storagereview.com/best/business-laptops)
- [Best Mobile Workstations](https://www.storagereview.com/best/mobile-workstations)
- [Best Desktop Workstations](https://www.storagereview.com/best/desktop-workstations)
- [Best Mini PCs](https://www.storagereview.com/best/mini-pcs)
- [Best Laptop Battery Life](https://www.storagereview.com/best/laptop-battery-life)
- [Storage Reference Guide](https://www.storagereview.com/storage-reference-guide)
- [About SR](https://www.storagereview.com/about-storagereview)
- [StorageReview.com Sweepstakes Rules and Regulations](https://www.storagereview.com/storagereview-com-sweepstakes-rules-and-regulations)
Search
[Home](https://www.storagereview.com/) » [News](https://www.storagereview.com/news) » NVIDIA Sets MLPerf Inference v6.0 Records with Blackwell Ultra Platform
# NVIDIA Sets MLPerf Inference v6.0 Records with Blackwell Ultra Platform
by Harold Fritts on April 1, 2026
[AI](https://www.storagereview.com/enterprise/ai) ◇ [Enterprise](https://www.storagereview.com/enterprise)
[NVIDIA has published results](https://developer.nvidia.com/blog/nvidia-extreme-co-design-delivers-new-mlperf-inference-records/) for MLPerf Inference v6.0, highlighting system-level gains driven by tight co-design across hardware, software, and models. The company positions inference throughput and token economics as the primary metrics for AI factory performance, moving beyond peak accelerator specifications to measured output under real workloads.
In this round, systems built on NVIDIA Blackwell Ultra GPUs delivered the highest throughput across all submitted models and scenarios. The ecosystem around the platform also expanded, with 14 partners submitting results, including major OEMs, cloud providers, and integrators such as ASUS, Cisco, CoreWeave, Dell Technologies, GigaComputing, Google Cloud, HPE, Lenovo, Nebius, Netweb Technology, QCT, Red Hat, Supermicro, and Lambda.
**Expanded Benchmark Coverage Reflects Emerging Workloads**
MLPerf Inference v6.0 introduces several new benchmarks to represent current AI deployments better. NVIDIA was the only vendor to submit across all new tests, spanning large language models, multimodal systems, generative video, and recommendation engines.
Key additions include DeepSeek-R1 Interactive, which evaluates higher interactivity with faster token delivery and reduced time to first token compared to prior server scenarios. The suite also adds Qwen3-VL-235B-A22B, marking the first multimodal vision-language model in MLPerf Inference, and GPT-OSS-120B, a mixture-of-experts reasoning model tested across offline, server, and interactive scenarios.
| Scenario | DeepSeek-R1 | GPT-OSS-120B | Qwen3-VL | Wan 2.2 | DLRMv3 |
| --- | --- | --- | --- | --- | --- |
| Offline | 2,494,310 tokens/sec\* | 1,046,150 tokens/sec | 79 samples/sec | 0.059 samples/sec | 104,637 samples/sec |
| Server | 1,555,110 tokens/sec\* | 1,096,770 tokens/sec | 68 queries/sec | 21 secs\*\*
(Single Stream) | 99,997 queries/sec |
| Interactive | 250,634 tokens/sec | 677,199 tokens/sec | \*\*\* | \*\*\* | \*\*\* |
\* Not a new scenario in MLPerf Inference v6.0
\*\* Wan 2.2 features a single stream scenario, which measures end-to-end request latency, instead of a server scenario. Lower is better.
\*\*\* Not tested in MLPerf Inference v6.0
Generative media and recommendation workloads are now included. The Wan 2.2 text-to-video model features both latency-sensitive and throughput-focused tests, while DLRMv3 replaces previous recommendation benchmarks with a transformer-based architecture that boosts compute intensity and model complexity.
**Software Optimization Drives Measurable Gains**
A notable aspect of this submission is the performance uplift achieved on existing hardware through software updates. NVIDIA reports up to 2.7x higher token throughput on the GB300 NVL72 platform for DeepSeek-R1 server scenarios compared to results from six months prior. This improvement translates to materially lower cost per token and higher utilization of deployed infrastructure.
![NVIDIA MLPerf v6 graphic ]()
These gains are attributed to updates in the TensorRT-LLM stack and associated frameworks. Kernel-level optimizations and fusion techniques reduce execution overhead, while improved attention data parallelism more effectively balances workloads across GPUs. Additional enhancements in the Dynamo distributed inference framework enable disaggregated serving, allowing independent optimization of prefill and decode phases.
For mixture-of-experts models, techniques like Wide Expert Parallel distribute expert weights across GPUs to reduce memory bottlenecks. Multi-token prediction boosts compute efficiency in low-batch, latency-sensitive scenarios by generating and validating multiple tokens at once. KV-aware routing further enhances scheduling by directing inference requests based on estimated compute costs.
| Benchmark | GB300 NVL72
v5.1 | GB300 NVL72
v6.0 | Speedup |
| --- | --- | --- | --- |
| DeepSeek-R1
(Server) | 2,907 tokens/sec/gpu | 8,064 tokens/sec/gpu | 2.77x |
| DeepSeek-R1
(Offline) | 5,842 tokens/sec/gpu | 9,821 tokens/sec/gpu | 1.68x |
| Llama 3.1 405B
(Server) | 170 tokens/sec/gpu | 259 tokens/sec/gpu | 1.52x |
| Llama 3.1 405B
(Offline) | 224 tokens/sec/gpu | 271 tokens/sec/gpu | 1.21x |
NVIDIA also demonstrated continued scaling on established models. On Llama 3.1 405B, the GB300 NVL72 platform achieved a 1.5x performance increase in server scenarios, indicating ongoing optimization for dense LLMs alongside newer architectures.
**Open Ecosystem and Framework Integration**
Submissions across new workloads leveraged a mix of NVIDIA and open-source frameworks. The Qwen3-VL benchmark used the vLLM framework, reflecting the rapid development in multimodal inference optimization. The Wan 2.2 text-to-video results were powered by TensorRT-LLM VisualGen, targeting diffusion-based pipelines on GPUs.
For DLRMv3, NVIDIA combined its recsys-example framework with GPU-accelerated embedding lookup technologies to handle the increased demands of transformer-based recommendation models. These integrations underscore the role of the broader software ecosystem in extracting performance from the underlying hardware.
**Scale-Out Performance with InfiniBand**
NVIDIA also showcased large-scale inference performance using four GB300 NVL72 systems connected via Quantum-X800 InfiniBand. This setup, with a total of 288 Blackwell Ultra GPUs, marks the largest MLPerf Inference submission to date and achieved system-level throughput of millions of tokens per second on DeepSeek-R1.
| DeepSeek-R1 \| 4x GB300 NVL72 | Tokens/Second |
| --- | --- |
| Offline | 2,494,310 |
| Server | 1,555,110 |
The results highlight the importance of high-performance interconnects in scaling inference workloads, particularly for distributed LLM serving and high-throughput batch processing.
**Toward Service-Level Benchmarking**
Looking ahead, NVIDIA is helping develop the MLPerf Endpoints within the MLCommons consortium. This upcoming benchmark aims to measure deployed inference services using real API traffic, giving insight into latency, throughput, and efficiency at the service level rather than just at the component level.
As AI workloads develop into agentic systems with longer context windows, benchmarks that measure end-to-end service performance are expected to become more important for both cloud providers and enterprise deployments.
**Engage with StorageReview**
[Newsletter](https://www.storagereview.com/storage_newsletter) | [YouTube](https://www.youtube.com/user/StorageReview "Opens in a new window") | Podcast [iTunes](https://podcasts.apple.com/gb/podcast/storagereview-com-storage-reviews/id1060681115 "Opens in a new window")/ [Spotify](https://open.spotify.com/show/1y6VnznABhHeOSMOmbDTz0 "Opens in a new window") | [Instagram](https://www.instagram.com/storagereview/ "Opens in a new window") | [Twitter](https://twitter.com/storagereview "Opens in a new window") | [TikTok](https://www.tiktok.com/@storagereview? "Opens in a new window") | [RSS Feed](https://www.storagereview.com/rss.xml)
![]()
### Harold Fritts
I have been in the tech industry since IBM created Selectric. My background, though, is writing. So I decided to get out of the pre-sales biz and return to my roots, doing a bit of writing but still being involved in technology.
Previous post: [Veeam Releases Open-Source MCP Server for Backup and Recovery Intelligence](https://www.storagereview.com/news/veeam-releases-open-source-mcp-server-for-backup-and-recovery-intelligence)
Next post: [Dell Technologies Enhances PowerProtect Portfolio for Improved Cyber Resilience](https://www.storagereview.com/news/dell-technologies-enhances-powerprotect-portfolio-for-improved-cyber-resilience)
Trusted Vendors
Products and solutions from our affiliate partners:
- [
Ubiquiti
](https://store.ui.com/us/en?a_aid=StorageReview "Opens in a new window")
- [
Newegg
](https://click.linksynergy.com/fs-bin/click?id=g5terNMDhz0&offerid=1207190.18&subid=0&type=4 "Opens in a new window")
- [
 Amazon
](https://amzn.to/4fLJUEW "Opens in a new window")
Newsletter
Subscribe to the StorageReview newsletter to stay up to date on the latest news and reviews. We promise no spam!
1
Leave this field empty if you’re human:
## Advertisement
Content Categories
[Facebook](https://www.facebook.com/@storagereview) [X](https://twitter.com/storagereview) [Instagram](https://www.instagram.com/storagereview/) [LinkedIn](https://www.linkedin.com/company/storagereview-com) [YouTube](https://www.youtube.com/user/storagereview?sub_confirmation=1) [Email](mailto:info@storagereview.com) [Spotify](https://open.spotify.com/show/1y6VnznABhHeOSMOmbDTz0) [Reddit](https://www.reddit.com/r/StorageReview/) [Discord](https://discord.gg/TwMHb4azdC) [RSS](https://www.storagereview.com/rss.xml) [TikTok](https://www.tiktok.com/@storagereview)
Copyright © 1998-2025 Flying Pig Ventures, LLC Cincinnati, Ohio. All rights reserved.
Manage your privacy
To provide the best experiences, we and our partners use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us and our partners to process personal data such as browsing behavior or unique IDs on this site and show (non-) personalized ads. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Click below to consent to the above or make granular choices. Your choices will be applied to this site only. You can change your settings at any time, including withdrawing your consent, by using the toggles on the Cookie Policy, or by clicking on the manage consent button at the bottom of the screen.
Functional Functional Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics Statistics
The technical storage or access that is used exclusively for statistical purposes. The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
Statistics
Marketing
Features
Always active
Special features
Special purposes
Always active
- Manage options
- Manage services
- Manage {vendor\_count} vendors
- [Read more about these purposes](https://cookiedatabase.org/tcf/purposes/)
Accept Deny Manage options Save preferences Manage options
- {title}
- {title}
- {title}
Manage your privacy
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Functional Functional Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics Statistics
The technical storage or access that is used exclusively for statistical purposes. The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
Statistics
Marketing
Features
Always active
Special features
Special purposes
Always active
- Manage options
- Manage services
- Manage {vendor\_count} vendors
- [Read more about these purposes](https://cookiedatabase.org/tcf/purposes/)
Accept Deny Manage options Save preferences Manage options
- {title}
- {title}
- {title}
Manage consent Manage consent
```json
{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.storagereview.com\/news\/nvidia-sets-mlperf-inference-v6-0-records-with-blackwell-ultra-platform","url":"https:\/\/www.storagereview.com\/news\/nvidia-sets-mlperf-inference-v6-0-records-with-blackwell-ultra-platform","name":"NVIDIA Sets MLPerf Inference v6.0 Records with Blackwell Ultra Platform - StorageReview.com","isPartOf":{"@id":"https:\/\/www.storagereview.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.storagereview.com\/news\/nvidia-sets-mlperf-inference-v6-0-records-with-blackwell-ultra-platform#primaryimage"},"image":{"@id":"https:\/\/www.storagereview.com\/news\/nvidia-sets-mlperf-inference-v6-0-records-with-blackwell-ultra-platform#primaryimage"},"thumbnailUrl":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2026\/04\/Storagereview-nvidia-end-to-end-graphic-mlperf-v6.jpg","datePublished":"2026-04-01T18:35:28+00:00","description":"NVIDIA Sets MLPerf Inference v6.0 Records with Blackwell Ultra Platform","breadcrumb":{"@id":"https:\/\/www.storagereview.com\/news\/nvidia-sets-mlperf-inference-v6-0-records-with-blackwell-ultra-platform#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.storagereview.com\/news\/nvidia-sets-mlperf-inference-v6-0-records-with-blackwell-ultra-platform"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.storagereview.com\/news\/nvidia-sets-mlperf-inference-v6-0-records-with-blackwell-ultra-platform#primaryimage","url":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2026\/04\/Storagereview-nvidia-end-to-end-graphic-mlperf-v6.jpg","contentUrl":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2026\/04\/Storagereview-nvidia-end-to-end-graphic-mlperf-v6.jpg","width":1384,"height":762,"caption":"NVIDIA MLPerf v6 graphic"},{"@type":"BreadcrumbList","@id":"https:\/\/www.storagereview.com\/news\/nvidia-sets-mlperf-inference-v6-0-records-with-blackwell-ultra-platform#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.storagereview.com\/"},{"@type":"ListItem","position":2,"name":"News","item":"https:\/\/www.storagereview.com\/news"},{"@type":"ListItem","position":3,"name":"NVIDIA Sets MLPerf Inference v6.0 Records with Blackwell Ultra Platform"}]},{"@type":"WebSite","@id":"https:\/\/www.storagereview.com\/#website","url":"https:\/\/www.storagereview.com\/","name":"StorageReview.com","description":"StorageReview.com is a leading provider of news and reviews throughout the entire IT stack - from the datacenter to the edge, and all points in between.","publisher":{"@id":"https:\/\/www.storagereview.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.storagereview.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.storagereview.com\/#organization","name":"StorageReview.com","url":"https:\/\/www.storagereview.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.storagereview.com\/#\/schema\/logo\/image\/","url":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2020\/02\/Storage-Reviews-2-2.png","contentUrl":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2020\/02\/Storage-Reviews-2-2.png","width":344,"height":61,"caption":"StorageReview.com"},"image":{"@id":"https:\/\/www.storagereview.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/storagereview","http:\/\/youtube.com\/user\/storagereview"]}]}
```