The AMD Instinct MI350P is AMD’s first new PCIe Instinct GPU in four years, following the MI210 in 2022. It brings the company’s CDNA 4 architecture to a conventional add-in card, with up to 4.6 PFLOPS of MXFP4 performance, 144GB of HBM3E, and 4TB/s of memory bandwidth. All of that fits into a dual-slot, 600W air-cooled card, giving customers another way to deploy Instinct GPUs without moving to the larger 8 GPU OAM MI350/5X platform.
Key Takeaways
- 144GB in a PCIe slot is the headline. The MI350P carries the most memory of any current PCIe accelerator, 3GB above the H200 NVL and 48GB above the RTX PRO 6000 Blackwell, with 4TB/s of HBM3E behind it, and ten of the fourteen model and precision combinations in this review ran on one card.
- Measured compute lands between the two NVIDIA cards. At 450W, MAMF reached 580 TFLOPS in BF16, 1,120 in FP8, and 2,228 in block-scaled FP4, 36 to 38 percent above a 600W RTX PRO 6000 at every precision and behind the H200 NVL in BF16 and FP8, which has no FP4 path.
- GPT-OSS is the MI350P’s strongest result. On AMDGPU 31.60, ROCm 10.1.0, and vLLM 0.32.0, GPT-OSS-120B reached 5,057 output tokens per second at 128 streams on the chat workload, 65 percent ahead of the RTX PRO 6000 and 50 percent ahead of the H200 NVL, and the curve keeps climbing through 128 streams on both prompt shapes.
- Chat-sized prompts favor the MI350P; long prompts are mixed. On 512-token prompts the card beats the RTX PRO 6000 on every FP8, BF16, and GPT-OSS model by 35 to 65 percent at 128 streams, and the two FP4-against-NVFP4 matchups are within 5 and 21 percent, while on 8,192-token prompts it leads on GPT-OSS and Mistral Small and trails on Llama, Qwen FP8, and MiniMax-M2.7, where its curve flattens early.
- The two-card results are on ROCm 7.2.3. DeepSeek-V4-Flash, MiniMax-M3, GLM-5.2, and GLM-5.3 Flash are reported from runs on that stack, because the ROCm 10.1 passes on those models used CPU expert offload and, on DeepSeek, a speculative decoder the NVIDIA runs did not.
- 600W is still the open question. Dell documents eight MI350Ps at 600W in the PowerEdge XE7745, and a follow-up at full power is planned.
AMD also showcased the MI350P at IFA 2026 in its Threadripper Halo Station prototype, where the same accelerators were liquid-cooled. The workstation concept supports up to four MI350Ps alongside a 96-core Threadripper PRO processor, providing 576GB of aggregate HBM3E capacity in a single machine; AMD’s product page, updated in October, now lists the system as coming in 2027. That gives us two ways to look at the MI350P: as a PCIe accelerator for servers, and as the building block for a larger deskside AI system competing with machines like the GB300 DGX Station.
This first look takes the server view, so the comparison that matters for a buyer specifying a PCIe accelerator is the NVIDIA RTX PRO 6000 Blackwell, the card that sits in the same power class and the same slots, and the H200 NVL is in the charts as the most capable NVIDIA add-in card for the workloads where memory bandwidth decides the result. A single GB300 DGX Station stands in for the deskside view; it is a different class of machine at a different price, and it is there to show how far a full Blackwell Ultra GPU sits above any of the PCIe cards.
AMD Instinct MI350P Specifications
AMD’s published numbers put the MI350P’s weight on low precision: 4.6 PFLOPS at MXFP4 and MXFP6, 2.3 PFLOPS at FP8, and 1.15 PFLOPS at FP16 or BF16, all dense figures. The memory system is the other half of the card: 144GB of HBM3E on a 4,096-bit interface at 4TB/s, behind a 128MB Infinity Cache, which is what the single-stream results below lean on. Board power is 600W with a 450W configurable mode; our review unit ran at 450W.
The MI350P uses the same CDNA 4 architecture as the larger, higher-power MI350/5X GPUs. The MI350P can be thought of as effectively half of a MI350X. To understand why, it helps to look at how the larger GPU is assembled.
The MI350X is a 3D-stacked chiplet design built on TSMC’s CoWoS, or Chip-on-Wafer-on-Substrate, packaging, with no single monolithic die. Eight accelerator complex dies, or XCDs, sit above two I/O dies, with four XCDs on each I/O die. AMD’s Infinity Fabric connects the chiplets, while eight HBM3E stacks provide the GPU’s memory. CoWoS describes the package-level integration; the compute-over-I/O arrangement is the design’s vertical, 3D-stacked component.
The MI350P takes one half of that arrangement: four XCDs, one I/O die, and four HBM3E stacks. Each XCD has 32 active compute units, giving the card 128 CUs compared with the MI350X’s 256. The memory resources are also halved: 144GB of HBM3E against 288GB, 4TB/s of bandwidth against 8TB/s, and 128MB of Infinity Cache against 256MB.
| Specification | AMD Instinct MI350P |
|---|---|
| Platform Overview | |
| Architecture | CDNA 4 |
| Compute Units | 128 |
| Stream Processors | 8,192 |
| Matrix Cores | 512 |
| Process | TSMC 3nm and 6nm |
| Transistors | 73 billion |
| Peak Engine Clock | 2,200MHz |
| Memory | |
| Capacity | 144GB HBM3E |
| Interface | 4,096-bit |
| Bandwidth | 4TB/s |
| Last Level Cache | 128MB |
| ECC | Full-chip |
| Peak Performance (AMD stated) | |
| MXFP4 | 4.6 PFLOPS |
| MXFP6 | 4.6 PFLOPS |
| MXFP8 | 2.3 PFLOPS |
| OCP FP8 | 2.3 PFLOPS; 4.6 PFLOPS with structured sparsity |
| FP16 | 1.15 PFLOPS; 2.3 PFLOPS with sparsity |
| BF16 | 1.15 PFLOPS; 2.3 PFLOPS with sparsity |
| FP32 | 72 TFLOPS |
| FP64 | 36 TFLOPS |
| Power, Cooling, and Form Factor | |
| Typical Board Power | 600W maximum; 450W configurable |
| Power Connector | 12V-2×6 |
| Cooling | Passive |
| Form Factor | PCIe add-in card, full height, double slot |
| Board Length | 10.5 in (267mm) |
| Host Interface | PCIe 5.0 x16 |
| Features | |
| SR-IOV | Yes |
| Infinity Cache | Yes |
| RAS | Yes |
| Page Retirement and Avoidance | Yes |
The 600W figure is a maximum, and AMD lets the card be configured down to 450W, which is where it ran in our 2U host. The H200 NVL matched it at 450W in the R770, but the RTX PRO 6000 ran at its full 600W in the XE7740, so the NVIDIA card had 150W more to work with in every comparison against it. The full-power results wait for a chassis that can cool a 600W passive card, and Dell’s XE7745 technical guide lists eight MI350Ps at 600W.
The other note: the single 12V-2×6 connector and the 10.5-inch, double-slot board, which is what lets the card go into the same risers that hold an RTX PRO 6000 or an H200 NVL today.
The table below sets the MI350P beside the three NVIDIA systems in our charts: the RTX PRO 6000 Blackwell Server Edition, which shares its slot, its power range, and its PCIe 5.0 x16 host link; the H200 NVL, the Hopper card with the memory bandwidth to match it; and the GB300 DGX Station, a single Blackwell Ultra GPU paired with a Grace CPU over NVLink-C2C.
The rows to read first are memory capacity, where the MI350P’s 144GB is the largest of the three PCIe cards, 3GB above the H200 NVL and 48GB above the RTX PRO 6000; memory bandwidth, where its 4TB/s sits between the two; and the FP4 line, where AMD quotes a dense figure, and NVIDIA quotes sparse.
| Specification | AMD Instinct MI350P | NVIDIA RTX PRO 6000 Blackwell Server Edition | NVIDIA H200 NVL PCIe | NVIDIA GB300 DGX Station |
|---|---|---|---|---|
| GPU architecture | CDNA 4 | Blackwell | Hopper | Blackwell Ultra |
| Form factor | PCIe accelerator | PCIe accelerator | PCIe accelerator | One Blackwell Ultra GPU paired with a 72-core Grace CPU |
| GPU memory | 144GB HBM3E | 96GB GDDR7 ECC | 141GB HBM3E | 252GB HBM3E |
| GPU memory bandwidth | 4TB/s | 1.6TB/s | 4.8TB/s | 7.1TB/s |
| FP4 | 4.6 PFLOPS MXFP4 (Dense) | 4 PFLOPS NVFP4 (Sparse) | No native FP4 | 20 PFLOPS NVFP4 (Sparse) |
| FP8 | 2.3 PFLOPS (Dense) | 2 PFLOPS (Sparse) | 3.3 PFLOPS (Sparse) | 10 PFLOPS (Sparse) |
| BF16 | 1.15 PFLOPS (Dense) | 1 PFLOPS (Sparse) | 1.6 PFLOPS (Sparse) | 5 PFLOPS (Sparse) |
| FP32 | 72 TFLOPS (Dense) | 120 TFLOPS (Sparse) | 60 TFLOPS (Sparse) | 80 TFLOPS (Sparse) |
| Host connection | PCIe 5.0 x16 | PCIe 5.0 x16 | PCIe 5.0 x16 | NVLink-C2C between Grace and GPU, 900GB/s (BiDi) |
| Multi-GPU connectivity | PCIe-based | PCIe-based | Two or four way NVLink, up to 900GB/s per GPU (BiDi) | 800Gb/s ConnectX-8 networking between Stations |
| Power | 450 to 600W configurable | 300 to 600W configurable | 450 to 600W configurable | Up to 1200W GPU with power sloshing |
Against the RTX PRO 6000, the MI350P’s memory subsystem is the most immediate difference. It has 50 percent more GPU memory and roughly 2.5x the memory bandwidth. For a model that doesn’t fit comfortably within 96GB, the additional capacity could determine whether it runs on one GPU or needs to be distributed across several.
The RTX PRO 6000, however, serves a broader range of workloads, with hardware for graphics, ray tracing, and video alongside its AI capabilities. The choice weighs more than two peak compute figures; it also depends on what else the system needs to do.
The H200 NVL is the more direct comparison for the MI350P among high-memory PCIe accelerators; the larger Blackwell datacenter GPUs sit in a different class. Memory capacity is nearly identical at 144GB versus the H200’s 141GB, with both using HBM3E, and the H200 offers 20 percent more memory bandwidth at 4.8TB/s. The two also scale differently: all three PCIe cards connect to the host over PCIe, but the H200 NVL adds dedicated GPU-to-GPU connectivity through NVLink bridges connecting up to four GPUs.
The larger MI350X and MI355X were built to compete with Blackwell, but the MI350P cuts that hardware configuration in half. That brings it closer to the H200 NVL in memory attributes and in overall positioning. One place where the MI350P still distinguishes itself is native MXFP4 and MXFP6 support, which matters more as inference moves toward lower precision.
The smaller memory subsystem is another compromise, because AMD has not offset the reduction in HBM stacks with faster HBM3E, leaving the MI350P at 4TB/s of bandwidth and behind the H200 NVL on this metric. That can limit performance in memory-bandwidth-bound workloads, regardless of the available compute, as the memory bandwidth and long-prompt inference results later in the review show.
The DGX Station comparison runs the other way, because the GB300 Station carries a full B300 GPU with one HBM stack disabled, a datacenter-class chip on a datacenter power budget, while the MI350P is half of a datacenter-class chip fitted into the 600W envelope of a PCIe add-in card. AMD needs two or four MI350Ps to meet that single GPU, and on paper two of them do: 288GB and 8TB/s against the Station’s 252GB and 7.1TB/s. Those resources sit on two cards, though, and a workload spanning both exchanges data over PCIe peer-to-peer, which limits how much the second card adds.
Performance
Test Platform
The MI350P review unit is a Dell PowerEdge R7725 with two AMD EPYC 9555 processors (64 cores each), 1.5TB of DDR5-6400 across 24 DIMMs, and two MI350P cards set to 450W, the cap for a 2U server. The RTX PRO 6000 Blackwell Server Edition cards (600W) ran in the Dell PowerEdge XE7740 that carries our four-card inference build, a single H200 NVL (450W) ran in the Dell PowerEdge R770 from our review, and the GB300 figures come from the ASUS ExpertCenter Pro ET900N G3 DGX Station we reviewed in September.
The iDRAC inventory below shows how the two cards are seated. They sit in PCIe slots 2 and 7, both Gen5 x16 and both reporting healthy, and the device keys, 0-139-0 and 1-199-0, read to us as one card behind each EPYC socket, so tensor-parallel traffic between the two MI350Ps crosses the socket link on its way between root complexes. That is the layout a two-card deployment in a 2U server gets by default, and it is worth keeping in mind on the two-card results later.
The memory view shows the rest of the host: 24 DIMMs of 64GB DDR5-6400, dual rank, for 1,536GB in a server that tops out at 6TB, all with multi-bit ECC. The headroom matters for the GLM-5.2 run, where the MXFP4 checkpoint does not fit in two cards’ 288GB and vLLM offloads part of the expert weights to host memory.
Maximum Achievable Matmul FLOPS (MAMF)
To put the MI350P’s compute in context, we ran MAMF across the four systems. MAMF (Maximum Achievable Matmul FLOPS) is a practical performance metric designed to measure the realistic peak floating-point operations per second an accelerator reaches during matrix multiplication, a more useful benchmark than the theoretical peak FLOPS in the spec sheet. We sweep each system in BF16, FP8, and block-scaled FP4 and report the best sustained dense result per precision; the hatched bars multiply each PCIe card’s result by the card count used in the large-model serving runs and are calculated, with no measurement behind them.
At 450W, one MI350P reaches 580 TFLOPS in BF16, 1,120 in FP8, and 2,228 in FP4. The RTX PRO 6000 Blackwell follows at 419, 820, and 1,634, so the MI350P holds a 36 to 38 percent lead over the NVIDIA card at every precision while running 150W below it, with the widest gap in BF16 at 38.4 percent.
The H200 NVL’s 652 TFLOPS in BF16 and 1,205 in FP8 put it ahead of the MI350P on both, and it has no FP4 path. The GB300 DGX Station sits above all three PCIe cards at 1,924, 3,802, and 6,024, roughly 3.3x the MI350P at BF16 and 2.7x at FP4; two MI350Ps on paper close to 1,160 and 4,456, still short of the Station’s single GPU. The NVIDIA MAMF figures in this review come from our latest runs on each card and replace the results from our earlier testing.
GPU Memory Bandwidth
Raw compute throughput is only useful if the accelerator can keep its execution units supplied with data. This matters most for inference, where token generation during the decode phase is limited more by memory bandwidth than by available FLOPS. The bandwidth test reads, writes, and copies GPU memory through compute kernels and the copy engine, showing the bandwidth available inside each GPU against its rated figure.
One MI350P reads its HBM3E at 3,291GB/s and writes at 3,204GB/s, about 80 percent of the 4TB/s rating, with copies at 1,276 to 1,283GB/s; a copy consumes bandwidth twice, once to read the source and again to write the destination. The RTX PRO 6000’s GDDR7 manages 1,540GB/s on reads and 1,483GB/s on writes, so the MI350P carries about 2.1x the usable bandwidth per card.
The H200 NVL’s 4,472GB/s read and 4,156GB/s write figures run 36 and 30 percent ahead of the MI350P, in line with its 4.8TB/s rating against 4TB/s, and the GB300 DGX Station reaches 6,869GB/s. Two MI350Ps measured together reach 6,529GB/s on reads, near the Station, but that bandwidth sits in two PCIe slots, and a model spanning both pays for every exchange over PCIe.
Inference
With the low-level tests covered, we move to LLM inference. Our vLLM online serving test drives an OpenAI-compatible endpoint with a fixed number of concurrent streams and records output tokens per second across the run. We report two shapes: 512 tokens in and 512 out, which is chat-sized and decode-heavy, and 8,192 in and 1,024 out, which is RAG- and document-sized and leans on prefill. Concurrency doubles from one stream upward, and each chart stops at the highest concurrency at which every card in it completed. Every point is two requests per stream, so single-stream figures rest on two requests and carry more run-to-run variance than the higher-concurrency points.
Each model ran on the fewest cards that could hold it: one MI350P for everything through GPT-OSS-120B and MiniMax-M2.7, and two for DeepSeek-V4-Flash, MiniMax-M3, GLM-5.2, and GLM-5.3 Flash. The RTX PRO 6000 runs follow the same rule, which puts two RTX cards against one MI350P on MiniMax-M2.7 and four against two on MiniMax-M3 and GLM-5.2. Low-precision builds differ by vendor: AMD’s are MXFP4 through Quark, NVIDIA’s are NVFP4, and the FP4 pairings are labeled that way throughout. GPT-OSS ships as MXFP4 and runs the same weights on every card. The H200 NVFP4 builds are not included.
| Model | MI350P | RTX PRO 6000 Blackwell | H200 NVL | GB300 DGX Station |
|---|---|---|---|---|
| GLM-5.3 Flash | 2 cards, MXFP4 | Not run | Not run | NVFP4 |
| GLM-5.2 | 2 cards, MXFP4 (offloads to host memory) | 4 cards, NVFP4 | Not run | NVFP4 (about 215GB of experts in Grace memory) |
| MiniMax-M3 | 2 cards, MXFP4 | 4 cards, NVFP4 | Not run | NVFP4 (a slice of experts in Grace memory) |
| DeepSeek-V4-Flash | 2 cards, FP8 | 2 cards, FP8 | Not run | FP8 |
| MiniMax-M2.7 | 1 card, MXFP4 | 2 cards, NVFP4 | Not run | NVFP4 |
| GPT-OSS-120B | 1 card, MXFP4 | 1 card, MXFP4 | 1 card, MXFP4 | MXFP4 |
| GPT-OSS-20B | 1 card, MXFP4 | 1 card, MXFP4 | 1 card, MXFP4 | MXFP4 |
| Qwen3-Coder-30B-A3B | 1 card, FP8 and BF16 | 1 card, FP8 and BF16 | 1 card, FP8 and BF16 | FP8 and BF16 |
| Mistral Small 3.1 24B | 1 card, FP8 and BF16 | 1 card, FP8 and BF16 | 1 card, FP8 and BF16 | FP8 and BF16 |
| Llama 3.1 8B | 1 card, MXFP4, FP8, and BF16 | 1 card, NVFP4, FP8, and BF16 | 1 card, FP8 and BF16 | NVFP4, FP8, and BF16 |
Read down the MI350P column, and the card count is the first thing 144GB gets you. Six of the ten models run on one MI350P, where the RTX PRO 6000’s 96GB needs a second card for MiniMax-M2.7, and the two MXFP4 checkpoints that stay on a pair of MI350Ps, MiniMax-M3 and GLM-5.2, need four RTX PRO 6000s. GLM-5.2 is the one that outgrows even 288GB, and vLLM offloads part of its expert weights to the R7725’s host memory for that run; the GB300 does the same with about 215GB of experts in Grace memory.
The MI350P figures for the single-card models were run October 27 on AMDGPU 31.60, ROCm 10.1.0, and vLLM 0.32.0. Dell’s validated combination is vLLM 0.27 with ROCm 10.0 and AMDGPU 31.5, and we went one step past it to the ROCm 10.1 release of October 5. The NVIDIA cards ran the vLLM 26.08 container on CUDA 13.4.1 throughout. The table lists the build behind each run.
| Model | vLLM Image |
|---|---|
| AMD Instinct MI350P | |
| DeepSeek-V4-Flash-0731 | vllm/vllm-openai-rocm@sha256:a78d3e91404e7875aa650a61dea580a9e26d72553bee9ae5b93dbdef7107594d (ROCm 7.2.3) |
| GLM-5.2 MXFP4 | vllm/vllm-openai-rocm:v0.27.1 (ROCm 7.2.3) |
| GLM-5.3 Flash MXFP4 | vllm/vllm-openai-rocm@sha256:a78d3e91404e7875aa650a61dea580a9e26d72553bee9ae5b93dbdef7107594d (ROCm 7.2.3) |
| GPT-OSS-120B MXFP4 | vLLM 0.32.0 on ROCm 10.1.0, AMDGPU 31.60 (October 27) |
| GPT-OSS-20B MXFP4 | vLLM 0.32.0 on ROCm 10.1.0, AMDGPU 31.60 (October 27) |
| Llama 3.1 8B BF16 | vLLM 0.32.0 on ROCm 10.1.0, AMDGPU 31.60 (October 27) |
| Llama 3.1 8B FP8 | vLLM 0.32.0 on ROCm 10.1.0, AMDGPU 31.60 (October 27) |
| Llama 3.1 8B MXFP4 | vLLM 0.32.0 on ROCm 10.1.0, AMDGPU 31.60 (October 27) |
| MiniMax-M2.7 MXFP4 | vLLM 0.32.0 on ROCm 10.1.0, AMDGPU 31.60 (October 27) |
| MiniMax-M3 MXFP4 | vllm/vllm-openai-rocm:nightly-2a02f6efe319c885e3ccbcecde402e0028f9ec1e (ROCm 7.2.3) |
| Mistral Small 3.1 24B BF16 | vLLM 0.32.0 on ROCm 10.1.0, AMDGPU 31.60 (October 27) |
| Mistral Small 3.1 24B FP8 | vLLM 0.32.0 on ROCm 10.1.0, AMDGPU 31.60 (October 27) |
| Qwen3-Coder-30B BF16 | vLLM 0.32.0 on ROCm 10.1.0, AMDGPU 31.60 (October 27) |
| Qwen3-Coder-30B FP8 | vLLM 0.32.0 on ROCm 10.1.0, AMDGPU 31.60 (October 27) |
| NVIDIA RTX PRO 6000 Blackwell Server Edition (CUDA 13.4.1.012) | |
| DeepSeek-V4-Flash-0731 | nvcr.io/nvidia/vllm:26.08-py3 |
| GLM-5.2 NVFP4 | nvcr.io/nvidia/vllm:26.08-py3 |
| GPT-OSS-120B MXFP4 | nvcr.io/nvidia/vllm:26.08-py3 |
| GPT-OSS-20B MXFP4 | nvcr.io/nvidia/vllm:26.08-py3 |
| Llama 3.1 8B BF16 | nvcr.io/nvidia/vllm:26.08-py3 |
| Llama 3.1 8B FP8 | nvcr.io/nvidia/vllm:26.08-py3 |
| Llama 3.1 8B NVFP4 | nvcr.io/nvidia/vllm:26.08-py3 |
| MiniMax-M2.7 NVFP4 | nvcr.io/nvidia/vllm:26.08-py3 |
| MiniMax-M3 NVFP4 | nvcr.io/nvidia/vllm:26.08-py3 |
| Mistral Small 3.1 24B BF16 | nvcr.io/nvidia/vllm:26.08-py3 |
| Mistral Small 3.1 24B FP8 | nvcr.io/nvidia/vllm:26.08-py3 |
| Qwen3-Coder-30B BF16 | nvcr.io/nvidia/vllm:26.08-py3 |
| Qwen3-Coder-30B FP8 | nvcr.io/nvidia/vllm:26.08-py3 |
| NVIDIA H200 NVL (CUDA 13.4.1.012) | |
| GPT-OSS-120B MXFP4 | nvcr.io/nvidia/vllm:26.08-py3 |
| GPT-OSS-20B MXFP4 | nvcr.io/nvidia/vllm:26.08-py3 |
| Llama 3.1 8B BF16 | nvcr.io/nvidia/vllm:26.08-py3 |
| Llama 3.1 8B FP8 | nvcr.io/nvidia/vllm:26.08-py3 |
| Mistral Small 3.1 24B BF16 | nvcr.io/nvidia/vllm:26.08-py3 |
| Mistral Small 3.1 24B FP8 | nvcr.io/nvidia/vllm:26.08-py3 |
| Qwen3-Coder-30B BF16 | nvcr.io/nvidia/vllm:26.08-py3 |
| Qwen3-Coder-30B FP8 | nvcr.io/nvidia/vllm:26.08-py3 |
The four two-card models, DeepSeek-V4-Flash, MiniMax-M3, GLM-5.2, and GLM-5.3 Flash, are reported from runs on ROCm 7.2.3. Our ROCm 10.1 passes on those models used CPU expert offload throughout and, on DeepSeek-V4-Flash, a speculative decoder the NVIDIA runs did not, which makes them a different test, so they are held for a clean pass on the current stack, and the tables label every MI350P row with its stack.
AMD’s preferred path for these models is its AIM containers; the Enterprise AI Suite 2.5 release of October 9 is validated on ROCm 10 and carries AIMs for GPT-OSS-120B, Llama 3.3 70B, and Gemma 3 27B, but AMD’s accelerator table lists an optimized MI350P profile only for Wan2.2 so far. Stock vLLM is therefore still the path an MI350P buyer takes today, and is what we measured.
Inference Results at a Glance
There are 28 charts below, so here is the short version in two tables: output tokens per second for each model at the highest concurrency every card completed, with the chat-sized 512-in, 512-out workload first. The MI350P figures are one card at 450W unless the row says otherwise, with a column for the stack each row comes from; the GB300 DGX Station is a single B300.
| Model | Cards (MI350P / RTX PRO 6000) | MI350P stack | Streams | MI350P | RTX PRO 6000 | H200 NVL | GB300 DGX Station |
|---|---|---|---|---|---|---|---|
| GLM-5.3 Flash (MXFP4 vs NVFP4) | 2 / none | ROCm 7.2.3 | 32 | 1,462 | Not run | Not run | 828 |
| GLM-5.2 (MXFP4 vs NVFP4) | 2 / 4 | ROCm 7.2.3 | 32 | 59 | 40 | Not run | 139 |
| MiniMax-M3 (MXFP4 vs NVFP4) | 2 / 4 | ROCm 7.2.3 | 32 | 1,509 | 759 | Not run | 1,041 |
| DeepSeek-V4-Flash | 2 / 2 | ROCm 7.2.3 | 32 | 1,081 | 829 | Not run | 1,766 |
| MiniMax-M2.7 (MXFP4 vs NVFP4) | 1 / 2 | ROCm 10.1 | 128 | 1,524 | 1,932 | Not run | 4,793 |
| GPT-OSS-120B MXFP4 | 1 / 1 | ROCm 10.1 | 128 | 5,057 | 3,064 | 3,376 | 9,280 |
| GPT-OSS-20B MXFP4 | 1 / 1 | ROCm 10.1 | 128 | 10,726 | 7,937 | 6,022 | 22,041 |
| Qwen3-Coder-30B FP8 | 1 / 1 | ROCm 10.1 | 128 | 6,451 | 3,995 | 7,459 | 12,439 |
| Qwen3-Coder-30B BF16 | 1 / 1 | ROCm 10.1 | 128 | 4,696 | 2,881 | 5,240 | 10,000 |
| Mistral Small 24B FP8 | 1 / 1 | ROCm 10.1 | 128 | 4,458 | 3,242 | 4,624 | 11,075 |
| Mistral Small 24B BF16 | 1 / 1 | ROCm 10.1 | 128 | 3,065 | 2,026 | 3,373 | 7,861 |
| Llama 3.1 8B (MXFP4 vs NVFP4) | 1 / 1 | ROCm 10.1 | 128 | 8,383 | 8,869 | Not run | 28,400 |
| Llama 3.1 8B FP8 | 1 / 1 | ROCm 10.1 | 128 | 11,045 | 8,073 | 11,054 | 25,602 |
| Llama 3.1 8B BF16 | 1 / 1 | ROCm 10.1 | 128 | 6,819 | 4,740 | 8,069 | 17,074 |
Two results stand out in the chat table: the MI350P beats the RTX PRO 6000 on every FP8, BF16, and GPT-OSS model, by 35 to 65 percent at 128 streams, trails NVIDIA’s NVFP4 builds only at the top of the Llama and MiniMax-M2.7 curves, and roughly doubles the NVIDIA card at a single stream on Llama 3.1 8B and Mistral Small 24B, where 4TB/s of HBM3E does the work.
The H200 NVL, with 20 percent more bandwidth, stays 4 to 18 percent ahead on the dense FP8 and BF16 models at 128 streams and ties on Llama 3.1 8B FP8; on the two GPT-OSS models the order reverses, and the MI350P finishes 50 and 78 percent ahead.
The second table repeats the exercise on the 8,192-token prompts, where prefill dominates the run and the balance between compute and memory bandwidth shifts toward the NVIDIA cards.
| Model | Cards (MI350P / RTX PRO 6000) | MI350P stack | Streams | MI350P | RTX PRO 6000 | H200 NVL | GB300 DGX Station |
|---|---|---|---|---|---|---|---|
| GLM-5.3 Flash (MXFP4 vs NVFP4) | 2 / none | ROCm 7.2.3 | 32 | 760 | Not run | Not run | 1,219 |
| GLM-5.2 (MXFP4 vs NVFP4) | 2 / 4 | ROCm 7.2.3 | 16 | 6 | 9 | Not run | 87 |
| MiniMax-M3 (MXFP4 vs NVFP4) | 2 / 4 | ROCm 7.2.3 | 8 | 524 | 444 | Not run | 103 |
| DeepSeek-V4-Flash | 2 / 2 | ROCm 7.2.3 | 32 | 621 | 345 | Not run | 954 |
| MiniMax-M2.7 (MXFP4 vs NVFP4) | 1 / 2 | ROCm 10.1 | 64 | 331 | 501 | Not run | 1,744 |
| GPT-OSS-120B MXFP4 | 1 / 1 | ROCm 10.1 | 128 | 2,101 | 1,169 | 1,825 | 5,023 |
| GPT-OSS-20B MXFP4 | 1 / 1 | ROCm 10.1 | 128 | 3,603 | 2,482 | 3,419 | 9,555 |
| Qwen3-Coder-30B FP8 | 1 / 1 | ROCm 10.1 | 128 | 759 | 823 | 1,820 | 3,837 |
| Qwen3-Coder-30B BF16 | 1 / 1 | ROCm 10.1 | 128 | 728 | 588 | 1,592 | 3,749 |
| Mistral Small 24B FP8 | 1 / 1 | ROCm 10.1 | 128 | 599 | 482 | 799 | 2,302 |
| Mistral Small 24B BF16 | 1 / 1 | ROCm 10.1 | 64 | 425 | 299 | 632 | 1,633 |
| Llama 3.1 8B (MXFP4 vs NVFP4) | 1 / 1 | ROCm 10.1 | 128 | 964 | 1,644 | Not run | 6,737 |
| Llama 3.1 8B FP8 | 1 / 1 | ROCm 10.1 | 128 | 1,211 | 1,481 | 2,045 | 6,164 |
| Llama 3.1 8B BF16 | 1 / 1 | ROCm 10.1 | 128 | 686 | 716 | 1,358 | 3,498 |
On prefill-heavy prompts, the picture is mixed: the MI350P leads the RTX PRO 6000 on both GPT-OSS models at every stream count and on Mistral Small, splits Qwen3-Coder at 128 streams (24 percent ahead on BF16, 8 percent behind on FP8), and flattens by 16 streams on Llama and MiniMax-M2.7, where it finishes behind the NVIDIA card. Against the H200, it leads on GPT-OSS by 15 percent on the 120B model and 5 percent on the 20B at 128 streams and trails by 25 to 58 percent on the dense models.
On the largest models, where two MI350Ps face four RTX PRO 6000s, the NVIDIA set leads GLM-5.2 through 16 streams on the chat workload with 384GB of GDDR7 against 288GB of HBM3E, while on MiniMax-M3 all three configurations stay within 10 percent through eight streams before the RTX quad pulls ahead at 16. The single GB300 clears every PCIe configuration by a wide margin on anything that fits in its 252GB.
GLM-5.3 Flash
GLM-5.3 Flash was not run on the RTX PRO 6000, so the comparison is two MI350P cards against the single GB300 Station through 32 streams. On the chat workload, the MI350P pair starts behind, 45 against 353 output tokens per second at one stream, and crosses the Station at eight streams on its way to 1,462 at 32, against 828 for the Station. On the prefill-heavy workload the Station leads throughout, 1,219 against 760 at 32 streams, where the pair’s prefill has to cross PCIe between the two cards and the B300 does it all in one piece of HBM.
GLM-5.2
GLM-5.2 is the memory story in this set. The MXFP4 checkpoint spills past the two MI350P cards’ 288GB into host memory, so single-stream throughput is 2 output tokens per second on the pair against 9 for four RTX PRO 6000 cards holding the NVFP4 build in 384GB of GDDR7 and 35 for the Station, which offloads about 215GB of expert weights to Grace memory. The RTX quad leads through 16 streams on the chat workload; at 32 the MI350P pair reaches 59 against 40, and the Station reaches 139. On prefill-heavy prompts, neither PCIe configuration gets past single digits through 16 streams, against 87 for the Station.
MiniMax-M3
Two MI350Ps hold the MXFP4 build of MiniMax-M3 in 288GB; the NVIDIA side needs four RTX PRO 6000 cards. The three configurations run within 10 percent of each other through eight streams on the chat workload; the RTX quad pulls ahead at 16 (1,181 against 998 for the pair and 940 for the Station), and at 32 the MI350P pair reads 1,509 against 759 for the quad and 1,041 for the Station.
On prefill-heavy prompts, the MI350P pair leads at every point through eight streams, 524 against 444 for the quad, while the Station, which keeps a slice of this model’s experts in Grace memory, falls to 103.
DeepSeek-V4-Flash
With two cards on each side and the native FP8 checkpoint, the RTX PRO 6000 pair leads through eight streams on the chat workload and the MI350P pair takes over from 16, finishing at 1,081 against 829 at 32 streams; the single Station reaches 1,766 at the same point. The prefill-heavy run follows the same shape, the MI350P pair at 621 against 345 for the RTX pair at 32 streams and 954 for the Station.
MiniMax-M2.7
One MI350P carries the MXFP4 build of MiniMax-M2.7 against two RTX PRO 6000 cards on NVFP4. The single MI350P leads the pair at one stream (124 against 115), stays 8 to 20 percent ahead through 64 streams 1,486 against 1,339, then flattens at 1,524 at 128 and 1,557 at 256, while the RTX pair continues to 1,932 and 2,842; the Station reaches 4,793.
On the prefill-heavy workload, the MI350P holds at 304 to 331 output tokens per second from eight streams through 64, against 501 for the RTX pair and 1,744 for the Station.
GPT-OSS-120B
GPT-OSS ships as MXFP4, so every card runs the same weights. One MI350P leads one RTX PRO 6000 at every stream count on both workloads, by 40 to 93 percent from four streams up, and finishes at 5,057 output tokens per second on the chat workload and 2,101 on prefill-heavy prompts at 128 streams, against 3,064 and 1,169 for the RTX PRO 6000 and 3,376 and 1,825 for the H200 NVL; the Station reaches 9,280 and 5,023.
AMD and Dell’s MLPerf Inference v6.1 submission of September 16 puts a ceiling on the same model: eight MI350Ps in a PowerEdge XE7745 produced 48,971 tokens per second offline and 40,868 in the server scenario on GPT-OSS-120B, which is 6,121 and 5,108 per GPU, on a tuned submission stack. That harness and ours are not like-for-like, but the 5,057 we measured at 128 streams on stock vLLM sits at the bottom of that range.
GPT-OSS-20B
The smaller OpenAI model follows the same curve. The MI350P leads the RTX PRO 6000 at every point, 8,174 against 4,992 at 64 streams and 10,726 against 7,937 at 128 on the chat workload, and 3,460 against 2,083 and 3,603 against 2,482 on prefill-heavy prompts, and it runs to 16,406 at 256 streams where the RTX PRO 6000 reads 11,077. The H200 NVL trails the MI350P on both shapes, at 6,022 and 3,419 at 128 streams, and the Station sits at 22,041 and 9,555.
Qwen3-Coder-30B-A3B
The 30B MoE with 3B active parameters gives the MI350P a lead over the RTX PRO 6000 at every concurrency on the chat workload, finishing at 6,451 against 3,995 on FP8 and 4,696 against 2,881 on BF16 at 128 streams, with the H200 at 7,459 and 5,240.
On prefill-heavy prompts, the FP8 run leads only through four streams and then trails the RTX PRO 6000 from eight through 128, 759 against 823 at the top, with the H200 at 1,820; the BF16 run climbs to 728 at 128 streams against 588 and 1,592.
Mistral Small 3.1 24B
Mistral Small is the MI350P’s strongest dense result against the RTX PRO 6000: 4,458 against 3,242 on FP8 and 3,065 against 2,026 on BF16 at 128 streams on the chat workload, and at one stream the MI350P roughly doubles the NVIDIA card (100 against 56 on FP8, 67 against 32 on BF16).
It is also the dense model where the MI350P leads on prefill-heavy prompts, 599 against 482 on FP8 at 128 streams and 425 against 299 on BF16 at 64, though the H200 reaches 799 and 632 at the same points.
Llama 3.1 8B
Llama 3.1 8B is here for continuity with our older GPU coverage. At 128 streams on the chat workload, the MI350P delivers 11,045 output tokens per second on FP8 against 8,073 for the RTX PRO 6000 and 11,054 for the H200, a tie with the Hopper card, and 6,819 on BF16 against 4,740 and 8,069. AMD’s MXFP4 build reaches 8,383 at 128 streams, 5 percent behind NVIDIA’s NVFP4 build on the RTX PRO 6000 (8,869), after leading it by 12 to 143 percent at every lower stream count; at a single stream the MXFP4 run reads 323 against 133.
The prefill-heavy runs invert the result: MI350P FP8 flattens by 16 streams and finishes at 1,211 against 1,481 for the RTX PRO 6000 and 2,045 for the H200, even though the MI350P starts 90 percent ahead of the RTX PRO 6000 at one stream, and the MXFP4 build trails NVFP4 by 41 percent at 128 streams.
Conclusion
The MI350P is the first Instinct card in four years that drops into a standard server, and at 450W in a 2U box it holds up as a PCIe accelerator: 144GB of HBM3E at 4TB/s, 1,120 TFLOPS of FP8 and 2,228 of block-scaled FP4 in MAMF against 820 and 1,634 for the RTX PRO 6000 Blackwell, and about 2.1x the NVIDIA card’s usable memory bandwidth.
Against the RTX PRO 6000, the MI350P was faster on chat-sized prompts on every FP8, BF16, and GPT-OSS model, by 35 to 65 percent at 128 streams, where HBM3E’s bandwidth and capacity beat a 96GB GDDR7 card, and it leads on both GPT-OSS models at every stream count on both prompt shapes.
The H200 NVL still beat it on the dense FP8 and BF16 tests, by 4 to 18 percent at 128 streams on chat prompts and by more on long prompts, with 20 percent more bandwidth and no FP4 path; on GPT-OSS the MI350P finished 50 and 78 percent ahead of the Hopper card. The larger models are less settled, since four RTX PRO 6000s lead the MI350P pair on GLM-5.2 through 16 streams, with 384GB against 288GB and more aggregate bandwidth and compute to bring to bear.
MiniMax-M3 is closer: the three configurations stay within 10 percent through eight streams on chat prompts, the RTX quad pulls ahead at 16, and the MI350P pair leads on long prompts through eight streams. Per card, the MI350P’s measured FP4 rate is 36 percent higher than the RTX PRO 6000’s, so the advantage is the card count. Those two-card results are on the ROCm 7.2.3 stack and will be run again on ROCm 10.1 alongside the 600W testing.
A definitive opinion on the MI350P is not possible for us yet without testing it at its full potential at 600W, which we plan to do in a PowerEdge XE7745; Dell supports eight MI350Ps at 600W in that chassis. We hear that AMD’s estimate for the extra 150W is about 10 percent more performance.
AMD has come a long way, and the MI350/5X line shows how capable the architecture is. We run an MI300X box in our lab for development, and the pace of the software stack showed in how much of the suite ran on stock vLLM without incident.
With what we saw in this setup, the MI350P is for buyers looking for an alternative to NVIDIA GPUs, either because of lead times, pricing, or to avoid relying on one manufacturer, and Dell’s support for it in the PowerEdge line gives that silicon diversity mainstream servers to fit the need.
To be fair, our work here is far from done. While many buyers may put these cards in 2U servers, there is more performance on the table when they go into a larger chassis designed for 600W cards. However they’re deployed, this is a great first look at AMD returning to the add-in card server market, and as the software stack continues to mature, the MI350P, and what comes next, has a promising future. Today, for a buyer whose models live in the 96GB to 144GB band, the MI350P is one of two PCIe cards that keep them on a single GPU, and the only one past 141GB.
Leaderboard: The AMD Instinct MI350P appears under Also Tested on our Best Enterprise GPUs leaderboard, with its two-card results under Scale-Out Configurations.
Amazon