---
description: NVIDIA continues to dominate in MLPerf Inference v5.1, but AMD is making steady advances with the MI300, closing the gap in efficiency.
title: "MLPerf Inference v5.1: NVIDIA Blackwell Ultra vs. AMD Instinct Platforms"
image: https://www.storagereview.com/wp-content/uploads/2025/09/deployment_tradeoffs.png
---

[![Storage Review]()](https://storagereview.com/)

---

[Facebook](https://www.facebook.com/@storagereview) [X (Twitter)](https://twitter.twitter.com/storagereview) [Instagram](https://www.instagram.com/storagereview/) [LinkedIn](https://www.linkedin.com/company/storagereview-com) [YouTube](https://www.youtube.com/user/storagereview?sub_confirmation=1) [Email](mailto:info@storagereview.com) [Spotify](https://open.spotify.com/show/1y6VnznABhHeOSMOmbDTz0) [Reddit](https://www.reddit.com/r/StorageReview/) [Discord](https://discord.gg/TwMHb4azdC) [RSS Feed](https://www.storagereview.com/rss.xml) [TikTok](https://www.tiktok.com/@storagereview)

[Facebook](https://www.facebook.com/@storagereview) [X (Twitter)](https://twitter.com/storagereview) [Instagram](https://www.instagram.com/storagereview/) [LinkedIn](https://www.linkedin.com/company/storagereview-com) [YouTube](https://www.youtube.com/user/storagereview?sub_confirmation=1) [Email](mailto:info@storagereview.com) [Spotify](https://open.spotify.com/show/1y6VnznABhHeOSMOmbDTz0) [Reddit](https://www.reddit.com/r/StorageReview/) [Discord](https://discord.gg/TwMHb4azdC) [RSS Feed](https://www.storagereview.com/rss.xml) [TikTok](https://www.tiktok.com/@storagereview)

[![StorageReview.com](https://www.storagereview.com/wp-content/uploads/2026/07/Storage-Review-2x.png)](https://www.storagereview.com)

≡ Menu

- [Home](https://www.storagereview.com/)
- [Storage Reviews](https://www.storagereview.com/review)
  - [Consumer Reviews](https://www.storagereview.com/consumer)
  - [Enterprise Reviews](https://www.storagereview.com/enterprise)
  - [Ubiquiti Reviews](https://www.storagereview.com/best/ubiquiti-reviews)
- [SR Merch](https://store.storagereview.com)
- [Leaderboards](https://www.storagereview.com/best)
  - [Best Storage Arrays](https://www.storagereview.com/best/storage-arrays)
  - [Best Enterprise SSDs](https://www.storagereview.com/best/enterprise-ssds)
  - [Best Servers](https://www.storagereview.com/best/servers)
  - [Best Desktops for Local AI](https://www.storagereview.com/best/desktops-local-ai)
  - [Best Laptops for Local AI](https://www.storagereview.com/best/laptops-local-ai)
  - [Best Local LLM Tools](https://www.storagereview.com/best/local-llm-tools)
  - [Agentic AI Hardware](https://www.storagereview.com/best/agentic-ai-hardware)
  - [Best SSDs & Hard Drives](https://www.storagereview.com/best_drives)
  - [Best Portable SSDs](https://www.storagereview.com/best/portable-ssds)
  - [Best Business Laptops](https://www.storagereview.com/best/business-laptops)
  - [Best Mobile Workstations](https://www.storagereview.com/best/mobile-workstations)
  - [Best Desktop Workstations](https://www.storagereview.com/best/desktop-workstations)
  - [Best Mini PCs](https://www.storagereview.com/best/mini-pcs)
  - [Best Laptop Battery Life](https://www.storagereview.com/best/laptop-battery-life)
- [Storage Reference Guide](https://www.storagereview.com/storage-reference-guide)
- [About SR](https://www.storagereview.com/about-storagereview)
  - [StorageReview.com Sweepstakes Rules and Regulations](https://www.storagereview.com/storagereview-com-sweepstakes-rules-and-regulations)

Search

[Home](https://www.storagereview.com/) » [News](https://www.storagereview.com/news) » MLPerf Inference v5.1: NVIDIA Blackwell Ultra vs. AMD Instinct Platforms

# MLPerf Inference v5.1: NVIDIA Blackwell Ultra vs. AMD Instinct Platforms

by Harold Fritts on September 9, 2025

[AI](https://www.storagereview.com/enterprise/ai)  ◇  [Enterprise](https://www.storagereview.com/enterprise)

MLPerf Inference v5.1 offers rigorous benchmarks for AI inference across LLMs, vision, and multimodal tasks. Here is the technical analysis on NVIDIA’s Blackwell Ultra results and AMD’s Instinct MI300X/MI325X/MI355X submissions. It includes detailed benchmark data, software optimizations, architectural strategies, and implications for hyperscalers and enterprises.

## MLPerf Benchmarking Framework

MLPerf Inference benchmarks serve as the scoreboard for AI accelerators, providing a standardized, apples-to-apples method for measuring performance across image classification, language models, and recommendation systems. For enterprises deploying GPUs at scale, these numbers often guide massive purchasing decisions.

MLPerf Inference is governed by MLCommons and defines Closed vs Open Division rules. Scenarios include:

- **Offline**: Throughput-centric (batch as many queries as possible).
- **Server**: Latency-compliant multi-stream inference.
- **Interactive**: Strict TTFT (Time-to-First-Token) and TPS (tokens per second @ 99th percentile).

The v5.1 workload suite included:

- **DeepSeek-R1:** A 671B mixture-of-experts model (stress test for reasoning inference).
- **Llama-3.1-405B and 8B**: Newest LLMs across multiple inference modes.
- **Whisper**: ASR replacing RNN-T.
- **Stable Diffusion XL (SD-XL)**: Text-to-image generative AI.
- **Mixtral**, **DLRMv2**, plus legacy workloads like ResNet-50.

When charts of throughput (tokens/sec or samples/sec) are shown, they typically highlight NVIDIA’s continued dominance in absolute numbers, especially per-GPU. However, AMD’s progress in relative efficiency and *scaling smoothness* shows they are closing the gap round over round, particularly in server scenarios and multi-node deployments.

## NVIDIA Blackwell Ultra Results

Blackwell Ultra systems achieved record throughput across all new workloads. Key enablers:

- NVFP4 precision: custom 4-bit float, accelerating DeepSeek and Llama.
- FP8 KV-cache: memory savings for attention layers.
- Disaggregated serving: split context vs generation stages across 72 GPUs via 1,800 GB/s NVLink.
- Software: CUDA Graphs, TensorRT-LLM, ADP Balance.

The bar chart below, contrasting Hopper vs Blackwell Ultra per-GPU, shows nearly 5× throughput uplift on DeepSeek-R1, validating that fine-grained data formats (NVFP4) and rack-wide bandwidth scale linearly for massive inference workloads. Another figure framed around interactive Llama-405B record-setting results demonstrates how NVIDIA uses NVLink fabric and disaggregated serving to keep latency SLAs while breaking past previous throughput limits. The takeaway: NVIDIA remains the industry pace-setter for sheer raw speed and latency-sensitive workloads.

![]()

### **AMD Instinct MI300X/MI325X/MI355X Results**

AMD targeted efficiency and flexibility:

- **FP4 on MI355X**: delivering 2.7× more Llama-2-70B throughput vs MI325X FP8.
- **Structured pruning**: On Llama-405B, pruning 21–33% reduced FLOPs without impacting accuracy, boosting throughput by \~82–90%.
- **Scaling**: Smooth linear scaling proven up to 8 nodes; first heterogeneous cluster (4 × MI300X + 2 × MI325X) achieved 94% scaling efficiency.
- **ROCm ecosystem reproducibility**: Partner submissions consistently within 1–3% of AMD’s own results.

![]()

The above chart showing MI325X FP8 vs. MI355X FP4 illustrates breakthrough FP4 benefits, higher tokens/sec with minimal accuracy loss, proving FP4 isn’t experimental but deployment-ready.

![]()

This **scaling curve** (1 → 8 nodes) chart highlights near-linear scaling, something hyperscalers value since it translates to predictable expansion costs. Meanwhile, the schematic of structured pruning shows AMD’s focus isn’t only on hardware brute force but also on algorithmic efficiency, crucial for real-world inference clusters constrained by power and space.

### **AMD-NVIDIA Head-to-Head Comparison**

| Metric | NVIDIA Blackwell Ultra | AMD MI355X | Notes | Winner |
| --- | --- | --- | --- | --- |
| Tokens/sec per GPU | 5842 (DeepSeek-R1) | \~2200 (scaled FP4) | Higher NVIDIA raw perf | NVIDIA |
| Memory per GPU | 192GB HBM3e | 288GB HBM3e | AMD fits 520B model single GPU | AMD |
| Precision support | FP16, FP8, NVFP4 | FP16, FP8, FP4 | Both 4-bit lead | Tie |
| Scaling Fabric | 72 GPU NVLink | Linear node scaling to 8 | Different scale strategies | Tie |

Side-by-side visualizations emphasize the trade-offs.

- NVIDIA dominates per-GPU throughput, making it ideal for AI factories where flop density matters.
- AMD, with larger memory (288GB) and FP4 pruning, shines where cost-per-token and deployment flexibility are paramount.

![]()

### **Industry Implications**

- **Hyperscalers**: NVIDIA remains the tool of choice for AI factories chasing peak performance. AMD’s growing efficiency innovations, however, unlock cost-optimized scaling, appealing for balancing workloads across tiers.
- **Cloud Providers**: Expect heterogeneous offerings, high-performance NVIDIA tiers vs. cost-efficient AMD Kubernetes pods.
- **Enterprises**: Structured pruning, FP4, and heterogeneous cluster support allow AMD to deliver 405B+ model inference at lower TCO. NVIDIA innovations (disaggregated serving and Dynamo) remain beneficial for latency-sensitive apps (e.g., real-time LLM chatbots).
- **Future Outlook**: Expect universal adoption of 4-bit inference as a baseline, broader use of heterogeneous GPU pools, and new system designs including NVIDIA’s Rubin CPX for long sequences and AMD’s ROCm expansion to cover more frameworks.

NVIDIA continues to set the pace in raw throughput, with H200 GPUs leading inference in ResNet-50 and BERT across closed division benchmarks. That said, AMD’s MI300 doesn’t trail far behind, posting competitive numbers in server and offline scenarios while also showing strong gains in power efficiency. The real story here isn’t just that NVIDIA remains on top, but that AMD has significantly closed the gap compared to just one MLPerf round ago. For buyers, this means the days of looking only at green GPUs may be over—ROCm is maturing, and MI300 is viable in real-world inference deployments.

**Engage with StorageReview**

[Newsletter](https://www.storagereview.com/storage_newsletter) |  [YouTube](https://www.youtube.com/user/StorageReview "Opens in a new window") | Podcast  [iTunes](https://podcasts.apple.com/gb/podcast/storagereview-com-storage-reviews/id1060681115 "Opens in a new window")/ [Spotify](https://open.spotify.com/show/1y6VnznABhHeOSMOmbDTz0 "Opens in a new window") |  [Instagram](https://www.instagram.com/storagereview/ "Opens in a new window") |  [Twitter](https://twitter.com/storagereview "Opens in a new window") |  [TikTok](https://www.tiktok.com/@storagereview? "Opens in a new window") |  [RSS Feed](https://www.storagereview.com/rss.xml)

![]()

### Harold Fritts

I have been in the tech industry since IBM created Selectric. My background, though, is writing. So I decided to get out of the pre-sales biz and return to my roots, doing a bit of writing but still being involved in technology.

Previous post: [Cisco, NVIDIA, and VAST Data Advance Agentic AI Infrastructure with Secure AI Factory Blueprint](https://www.storagereview.com/news/cisco-nvidia-and-vast-data-advance-agentic-ai-infrastructure-with-secure-ai-factory-blueprint)

Next post: [NVIDIA Unveils Roadmap at AI Infra Summit: From Blackwell Ultra to Vera Rubin CPX Architecture](https://www.storagereview.com/news/nvidia-unveils-roadmap-at-ai-infra-summit-from-blackwell-ultra-to-vera-rubin-cpx-architecture)

Trusted Vendors

Products and solutions from our affiliate partners:

- [  
  ![Ubiquiti Logo](https://www.storagereview.com/wp-content/uploads/2024/11/2-1.webp)  
   Ubiquiti  
  ](https://store.ui.com/us/en?a_aid=StorageReview "Opens in a new window")
- [  
  ![Newegg Logo](https://www.storagereview.com/wp-content/uploads/2025/11/newegg.webp)  
   Newegg  
  ](https://click.linksynergy.com/fs-bin/click?id=g5terNMDhz0&offerid=1207190.18&subid=0&type=4 "Opens in a new window")
- [  
  ![Amazon Logo](https://www.storagereview.com/wp-content/uploads/2025/11/amazon.webp) Amazon  
  ](https://amzn.to/4fLJUEW "Opens in a new window")

Newsletter

Subscribe to the StorageReview newsletter to stay up to date on the latest news and reviews. We promise no spam!

1  

Leave this field empty if you’re human:

## Advertisement

Content Categories

[Facebook](https://www.facebook.com/@storagereview) [X](https://twitter.com/storagereview) [Instagram](https://www.instagram.com/storagereview/) [LinkedIn](https://www.linkedin.com/company/storagereview-com) [YouTube](https://www.youtube.com/user/storagereview?sub_confirmation=1) [Email](mailto:info@storagereview.com) [Spotify](https://open.spotify.com/show/1y6VnznABhHeOSMOmbDTz0) [Reddit](https://www.reddit.com/r/StorageReview/) [Discord](https://discord.gg/TwMHb4azdC) [RSS](https://www.storagereview.com/rss.xml) [TikTok](https://www.tiktok.com/@storagereview)

Copyright © 1998-2025 Flying Pig Ventures, LLC Cincinnati, Ohio. All rights reserved.

Manage your privacy

To provide the best experiences, we and our partners use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us and our partners to process personal data such as browsing behavior or unique IDs on this site and show (non-) personalized ads. Not consenting or withdrawing consent, may adversely affect certain features and functions.

Click below to consent to the above or make granular choices. Your choices will be applied to this site only. You can change your settings at any time, including withdrawing your consent, by using the toggles on the Cookie Policy, or by clicking on the manage consent button at the bottom of the screen.

Functional Functional Always active

The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.

Preferences Preferences

The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.

Statistics Statistics

The technical storage or access that is used exclusively for statistical purposes. The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.

Marketing Marketing

The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.

Statistics

Marketing

Features

Always active

Always active

- Manage options
- Manage services
- Manage {vendor\_count} vendors
- [Read more about these purposes](https://cookiedatabase.org/tcf/purposes/)

Accept Deny Manage options Save preferences Manage options

- {title}
- {title}
- {title}

Manage your privacy

To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.

Functional Functional Always active

The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.

Preferences Preferences

The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.

Statistics Statistics

The technical storage or access that is used exclusively for statistical purposes. The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.

Marketing Marketing

The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.

Statistics

Marketing

Features

Always active

Always active

- Manage options
- Manage services
- Manage {vendor\_count} vendors
- [Read more about these purposes](https://cookiedatabase.org/tcf/purposes/)

Accept Deny Manage options Save preferences Manage options

- {title}
- {title}
- {title}

Manage consent Manage consent

```json
{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.storagereview.com\/news\/mlperf-inference-v5-1-nvidia-blackwell-ultra-vs-amd-instinct-platforms","url":"https:\/\/www.storagereview.com\/news\/mlperf-inference-v5-1-nvidia-blackwell-ultra-vs-amd-instinct-platforms","name":"MLPerf Inference v5.1: NVIDIA Blackwell Ultra vs. AMD Instinct Platforms - StorageReview.com","isPartOf":{"@id":"https:\/\/www.storagereview.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.storagereview.com\/news\/mlperf-inference-v5-1-nvidia-blackwell-ultra-vs-amd-instinct-platforms#primaryimage"},"image":{"@id":"https:\/\/www.storagereview.com\/news\/mlperf-inference-v5-1-nvidia-blackwell-ultra-vs-amd-instinct-platforms#primaryimage"},"thumbnailUrl":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2025\/09\/deployment_tradeoffs.png","datePublished":"2025-09-09T19:35:29+00:00","description":"NVIDIA continues to dominate in MLPerf Inference v5.1, but AMD is making steady advances with the MI300, closing the gap in efficiency.","breadcrumb":{"@id":"https:\/\/www.storagereview.com\/news\/mlperf-inference-v5-1-nvidia-blackwell-ultra-vs-amd-instinct-platforms#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.storagereview.com\/news\/mlperf-inference-v5-1-nvidia-blackwell-ultra-vs-amd-instinct-platforms"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.storagereview.com\/news\/mlperf-inference-v5-1-nvidia-blackwell-ultra-vs-amd-instinct-platforms#primaryimage","url":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2025\/09\/deployment_tradeoffs.png","contentUrl":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2025\/09\/deployment_tradeoffs.png","width":1500,"height":900},{"@type":"BreadcrumbList","@id":"https:\/\/www.storagereview.com\/news\/mlperf-inference-v5-1-nvidia-blackwell-ultra-vs-amd-instinct-platforms#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.storagereview.com\/"},{"@type":"ListItem","position":2,"name":"News","item":"https:\/\/www.storagereview.com\/news"},{"@type":"ListItem","position":3,"name":"MLPerf Inference v5.1: NVIDIA Blackwell Ultra vs. AMD Instinct Platforms"}]},{"@type":"WebSite","@id":"https:\/\/www.storagereview.com\/#website","url":"https:\/\/www.storagereview.com\/","name":"StorageReview.com","description":"StorageReview.com is a leading provider of news and reviews throughout the entire IT stack - from the datacenter to the edge, and all points in between.","publisher":{"@id":"https:\/\/www.storagereview.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.storagereview.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.storagereview.com\/#organization","name":"StorageReview.com","url":"https:\/\/www.storagereview.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.storagereview.com\/#\/schema\/logo\/image\/","url":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2020\/02\/Storage-Reviews-2-2.png","contentUrl":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2020\/02\/Storage-Reviews-2-2.png","width":344,"height":61,"caption":"StorageReview.com"},"image":{"@id":"https:\/\/www.storagereview.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/storagereview","http:\/\/youtube.com\/user\/storagereview"]}]}
```