---
description: Supermicro, IBM, and NVIDIA pair Petascale E3.S nodes, Storage Scale, and Dynamo in a KV cache tier claiming 56x faster TTFT at 130K tokens
title: Supermicro KV Cache Platform With IBM Storage Scale and NVIDIA Dynamo Claims 56x Faster TTFT at 130K Tokens
image: https://www.storagereview.com/wp-content/uploads/2026/11/Supermicro-ASG-1115S-NE316R-Petascale-1U-E3S.jpg
---
[![Storage Review]()](https://storagereview.com/)
---
[Facebook](https://www.facebook.com/@storagereview) [X (Twitter)](https://twitter.twitter.com/storagereview) [Instagram](https://www.instagram.com/storagereview/) [LinkedIn](https://www.linkedin.com/company/storagereview-com) [YouTube](https://www.youtube.com/user/storagereview?sub_confirmation=1) [Email](mailto:info@storagereview.com) [Spotify](https://open.spotify.com/show/1y6VnznABhHeOSMOmbDTz0) [Reddit](https://www.reddit.com/r/StorageReview/) [Discord](https://discord.gg/TwMHb4azdC) [RSS Feed](https://www.storagereview.com/rss.xml) [TikTok](https://www.tiktok.com/@storagereview)
[Facebook](https://www.facebook.com/@storagereview) [X (Twitter)](https://twitter.com/storagereview) [Instagram](https://www.instagram.com/storagereview/) [LinkedIn](https://www.linkedin.com/company/storagereview-com) [YouTube](https://www.youtube.com/user/storagereview?sub_confirmation=1) [Email](mailto:info@storagereview.com) [Spotify](https://open.spotify.com/show/1y6VnznABhHeOSMOmbDTz0) [Reddit](https://www.reddit.com/r/StorageReview/) [Discord](https://discord.gg/TwMHb4azdC) [RSS Feed](https://www.storagereview.com/rss.xml) [TikTok](https://www.tiktok.com/@storagereview)
[](https://www.storagereview.com)
≡ Menu
- [Home](https://www.storagereview.com/)
- [Storage Reviews](https://www.storagereview.com/review)
- [End User Reviews](https://www.storagereview.com/consumer)
- [Enterprise Reviews](https://www.storagereview.com/enterprise)
- [Ubiquiti Reviews](https://www.storagereview.com/best/ubiquiti-reviews)
- [SR Merch](https://store.storagereview.com)
- [Leaderboards](https://www.storagereview.com/best)
- [Best Storage Arrays](https://www.storagereview.com/best/storage-arrays)
- [Best Enterprise SSDs](https://www.storagereview.com/best/enterprise-ssds)
- [Best Servers](https://www.storagereview.com/best/servers)
- [Best Enterprise GPUs](https://www.storagereview.com/best/enterprise-gpus)
- [Best Desktops for Local AI](https://www.storagereview.com/best/desktops-local-ai)
- [Best Laptops for Local AI](https://www.storagereview.com/best/laptops-local-ai)
- [Best Local LLM Tools](https://www.storagereview.com/best/local-llm-tools)
- [Agentic AI Hardware](https://www.storagereview.com/best/agentic-ai-hardware)
- [Best SSDs & Hard Drives](https://www.storagereview.com/best_drives)
- [Best Portable SSDs](https://www.storagereview.com/best/portable-ssds)
- [Best Business Laptops](https://www.storagereview.com/best/business-laptops)
- [Best Mobile Workstations](https://www.storagereview.com/best/mobile-workstations)
- [Best Desktop Workstations](https://www.storagereview.com/best/desktop-workstations)
- [Best Mini PCs](https://www.storagereview.com/best/mini-pcs)
- [Best Laptop Battery Life](https://www.storagereview.com/best/laptop-battery-life)
- [Storage Reference Guide](https://www.storagereview.com/storage-reference-guide)
- [About SR](https://www.storagereview.com/about-storagereview)
- [StorageReview.com Sweepstakes Rules and Regulations](https://www.storagereview.com/storagereview-com-sweepstakes-rules-and-regulations)
Search
[Home](https://www.storagereview.com/) » [News](https://www.storagereview.com/news) » Supermicro KV Cache Platform With IBM Storage Scale and NVIDIA Dynamo Claims 56x Faster TTFT at 130K Tokens
# Supermicro KV Cache Platform With IBM Storage Scale and NVIDIA Dynamo Claims 56x Faster TTFT at 130K Tokens
by Brian Beeler on November 5, 2026
[AI](https://www.storagereview.com/enterprise/ai) ◇ [Enterprise](https://www.storagereview.com/enterprise)
Supermicro announced a KV cache storage solution with IBM and NVIDIA today that moves the key-value state behind long-context inference off the GPU and onto a shared all-flash tier. The package combines Supermicro Petascale storage servers on AMD EPYC 9005 processors, IBM Storage Scale, NVIDIA Dynamo, and NVIDIA Spectrum-X Ethernet, and the companies back it with a validated reference architecture that IBM published as a Redbook. The headline figure is a 56x cut in time to first token on 130,000-token prompts against a GPU that recomputes the context from scratch.
## Why KV Cache Needs a Storage Tier
Inference runs in two phases: prefill turns the prompt into tokens and pushes them through the model to produce the intermediate key-value states, and decode generates the response one token at a time against those states. Keeping the KV state in a persistent cache lets the system reuse it for the next turn, the next agent step, or the next request that shares the same context, and skip the recompute.
Agentic workflows, coding assistants, long-context RAG, and multi-turn applications generate enough of this state to overflow GPU and CPU memory quickly, which is why Supermicro, IBM, and NVIDIA are making a network storage tier.
In this design, NVIDIA Dynamo manages the cache hierarchy and moves blocks over NIXL; IBM Storage Scale Erasure Code Edition provides the shared file tier; and the Petascale servers supply the flash and the PCIe Gen5 lanes behind it. Supermicro says the three companies tested the stack at cluster level under high concurrency and with noisy-neighbor network traffic, and it is pitching the result as a turnkey build with factory rack integration and on-site deployment that it says cuts deployment time from weeks to days.
## The Tested Configuration
The reference architecture uses eight Supermicro ASG-1115S-NE316R Petascale nodes, each a 1U system with a single AMD EPYC 9535 and 16 Micron 7.68TB E3.S NVMe drives, which puts 122.88TB of raw flash in each node and about 983TB across the eight. Each node carries a dual-port 200Gb NVIDIA BlueField-3 B3220 on the client side and a 400Gb ConnectX-7 for the storage backend, with three Spectrum-X SN5600 switches in a spine and leaf layout and 800Gb/s uplinks.
The storage nodes run RHEL 9.6 and IBM Storage Scale ECE 6.0.0.1 in an 8+2P erasure code layout, and IBM puts the read throughput of this eight-node tier at 315GB/s from earlier testing.
| **Component** | **KV Cache Reference Architecture (as tested)** |
| --- | --- |
| **Storage Tier** | |
| **Storage nodes** | 8x Supermicro ASG-1115S-NE316R (1U, single socket) |
| **CPU per node** | AMD EPYC 9535, 128 lanes of PCIe Gen5 (64 to NVMe, 64 to networking) |
| **Flash per node** | 16x Micron 7.68TB E3.S NVMe (122.88TB raw) |
| **Network per node** | NVIDIA BlueField-3 B3220, dual-port 200Gb (client side)
NVIDIA ConnectX-7 400Gb (storage backend) |
| **Switching** | 3x NVIDIA Spectrum-X SN5600, spine and leaf, 800Gb/s uplinks |
| **Storage software** | IBM Storage Scale Erasure Code Edition 6.0.0.1, 8+2P
RHEL 9.6 |
| **Inference Client** | |
| **System** | Supermicro SYS-212GB-FNR (NVIDIA MGX), Ubuntu 24.04 |
| **GPU used for the benchmarks** | 1x NVIDIA RTX PRO 6000 Blackwell, 19.6GiB left for KV cache after weights and CUDA graphs |
| **Client networking** | 2x NVIDIA BlueField-3 B3140H 400Gb/s, up to 80GB/s client-facing bandwidth |
| **Model and software** | openai/gpt-oss-120b (MXFP4)
vLLM 0.14.1
NVIDIA Dynamo 0.9.0 with NIXL
NVIDIA AIPerf benchmark client |
| **Cache tiering** | Host and local tiers set to zero; all 1.4 million cached tokens served from the Storage Scale tier |
The benchmark client is a single Supermicro SYS-212GB-FNR MGX system, and results come from one NVIDIA RTX PRO 6000 Blackwell GPU in that box running the openai/gpt-oss-120b model in MXFP4 under vLLM 0.14.1 and Dynamo 0.9.0.
After the weights and CUDA graphs, 19.6GiB of GPU memory was left for KV cache, enough for about 1.14 million tokens on the GPU itself. For the tests, the host memory and local tiers were set to zero so that every cached block came back from Storage Scale, so the results measure the storage path by itself, with no help from a mixed hierarchy.
## 56x on Time to First Token, 22x on Throughput
With no cache, time to first token for the 120B model rose from 0.572 seconds at 10,000 input tokens to 32.14 seconds at 130,000 as the GPU recomputed the full context. With the KV cache on Storage Scale, TTFT stayed between 0.193 and 0.57 seconds across the same range, which is the 56x figure at the 130,000-token mark and the 0.377-second spread Supermicro cites in its announcement. IBM, Supermicro, and NVIDIA ran these tests, and we did not run or audit the results.
The throughput test sent 200 requests over 28 concurrent connections, drawn from 100 unique prompts totaling 24 million tokens and 825GB of KV cache data, with the cache starting empty. The recompute baseline finished at 0.19 requests per second in 1,048 seconds; the Storage Scale tier finished at 4.26 requests per second in 46.94 seconds, a 22x gain that includes the cold-start cost of filling the cache.
A third run added four clients pushing 200GB/s of unrelated network traffic across the same fabric, and the cached configuration held 3.6 requests per second and 55.56 seconds for the 200 requests, about nine seconds slower than the clean run and still 18x the baseline. IBM attributes the small penalty to the Spectrum-X fabric isolating the cache traffic from the background load.
“Agentic AI and long-context applications depend on reusing context efficiently across many steps, and NVIDIA Dynamo helps orchestrate distributed inference so enterprises can get more usable throughput from every GPU,” said Shruti Koparkar, NVIDIA AI Inference Product Lead. “By pairing NVIDIA Dynamo and Spectrum-X Ethernet with Supermicro Petascale storage servers and IBM Storage Scale, this KV cache solution creates a high-performance shared storage tier that can reduce redundant computation, improve time-to-first-token and help customers scale resilient AI inference services.”
## Sizing and What Is Not in the Numbers
IBM’s sizing guide puts a four-node build at 4+2P with 67 percent usable capacity for 7B to 34B models and up to 64 GPUs, the tested eight-node build at 8+2P and 80 percent usable for 70B to 120B models and 100 to 256 GPUs needing 100 to 300GB/s of read bandwidth, and 16 or more nodes at 16+3P and 84 percent usable for larger models and 300 or more GPUs at 300GB/s and up. Storage Scale ECE itself scales from three storage servers to 256.
The single-GPU test setup is the main caveat. The reference architecture is sized for hundreds of GPUs, but every result in the paper comes from one RTX PRO 6000 against an eight-node, 983TB flash tier that IBM rates at 315GB/s, so the storage side was nowhere near saturation, and the tests do not show how TTFT holds up when a full GPU cluster is pulling cache at once.
Supermicro has not published pricing or an availability date for the packaged solution, and the announcement gives no GPU utilization figures. The approach lands in the same territory as [DDN’s Dynamo-integrated KV cache acceleration on the AI400X3M](https://www.storagereview.com/news/ddn-launches-ai400x3m-and-dynamo-integrated-kv-cache-acceleration-at-isc-2026), with the difference that this stack runs on commodity Supermicro servers and Ethernet under IBM’s file system.
“As AI workloads expand from training to inference, enterprises and neoclouds deploying high-throughput inference workloads characteristic of agentic AI and long-context, multi-turn generative AI need to remove bottlenecks to improve GPU efficiency,” said Charles Liang, president and CEO of Supermicro. “Our solution with IBM and NVIDIA clearly demonstrates these performance benefits.”
### [Supermicro ASG-1115S-NE316R Petascale Storage Server](https://www.supermicro.com/en/products/system/storage/1u/asg-1115s-ne316r)
### [IBM Storage Scale](https://www.ibm.com/products/storage-scale)
**Engage with StorageReview**
[Newsletter](https://www.storagereview.com/storage_newsletter) | [YouTube](https://www.youtube.com/user/StorageReview "Opens in a new window") | Podcast [iTunes](https://podcasts.apple.com/gb/podcast/storagereview-com-storage-reviews/id1060681115 "Opens in a new window")/ [Spotify](https://open.spotify.com/show/1y6VnznABhHeOSMOmbDTz0 "Opens in a new window") | [Instagram](https://www.instagram.com/storagereview/ "Opens in a new window") | [Twitter](https://twitter.com/storagereview "Opens in a new window") | [TikTok](https://www.tiktok.com/@storagereview? "Opens in a new window") | [RSS Feed](https://www.storagereview.com/rss.xml)
![]()
### Brian Beeler
Brian is located in Cincinnati, Ohio and is the chief analyst and President of StorageReview.com.
Previous post: [IBM, ORNL, and AMD Co-Locate the 120-Qubit Nighthawk With an MI355X Umbra Cluster for Quantum Error Correction](https://www.storagereview.com/news/ibm-ornl-amd-quantum-error-correction-co-design-platform-nighthawk-umbra-mi355x)
Trusted Vendors
Products and solutions from our affiliate partners:
- [
Ubiquiti
](https://store.ui.com/us/en?a_aid=StorageReview "Opens in a new window")
- [
Newegg
](https://click.linksynergy.com/fs-bin/click?id=g5terNMDhz0&offerid=1207190.18&subid=0&type=4 "Opens in a new window")
- [
 Amazon
](https://amzn.to/4fLJUEW "Opens in a new window")
Newsletter
Subscribe to the StorageReview newsletter to stay up to date on the latest news and reviews. We promise no spam!
1
Leave this field empty if you’re human:
## Advertisement
Content Categories
[Facebook](https://www.facebook.com/@storagereview) [X](https://twitter.com/storagereview) [Instagram](https://www.instagram.com/storagereview/) [LinkedIn](https://www.linkedin.com/company/storagereview-com) [YouTube](https://www.youtube.com/user/storagereview?sub_confirmation=1) [Email](mailto:info@storagereview.com) [Spotify](https://open.spotify.com/show/1y6VnznABhHeOSMOmbDTz0) [Reddit](https://www.reddit.com/r/StorageReview/) [Discord](https://discord.gg/TwMHb4azdC) [RSS](https://www.storagereview.com/rss.xml) [TikTok](https://www.tiktok.com/@storagereview)
Copyright © 1998-2025 Flying Pig Ventures, LLC Cincinnati, Ohio. All rights reserved.
Manage your privacy
To provide the best experiences, we and our partners use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us and our partners to process personal data such as browsing behavior or unique IDs on this site and show (non-) personalized ads. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Click below to consent to the above or make granular choices. Your choices will be applied to this site only. You can change your settings at any time, including withdrawing your consent, by using the toggles on the Cookie Policy, or by clicking on the manage consent button at the bottom of the screen.
Functional Functional Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics Statistics
The technical storage or access that is used exclusively for statistical purposes. The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
Statistics
Marketing
Features
Always active
Special features
Special purposes
Always active
- Manage options
- Manage services
- Manage {vendor\_count} vendors
- [Read more about these purposes](https://cookiedatabase.org/tcf/purposes/)
Accept Deny Manage options Save preferences Manage options
- {title}
- {title}
- {title}
Manage your privacy
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Functional Functional Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics Statistics
The technical storage or access that is used exclusively for statistical purposes. The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
Statistics
Marketing
Features
Always active
Special features
Special purposes
Always active
- Manage options
- Manage services
- Manage {vendor\_count} vendors
- [Read more about these purposes](https://cookiedatabase.org/tcf/purposes/)
Accept Deny Manage options Save preferences Manage options
- {title}
- {title}
- {title}
Manage consent Manage consent
```json
{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.storagereview.com\/news\/supermicro-ibm-storage-scale-nvidia-dynamo-kv-cache-platform-56x-ttft","url":"https:\/\/www.storagereview.com\/news\/supermicro-ibm-storage-scale-nvidia-dynamo-kv-cache-platform-56x-ttft","name":"Supermicro KV Cache Platform With IBM Storage Scale and NVIDIA Dynamo Claims 56x Faster TTFT at 130K Tokens - StorageReview.com","isPartOf":{"@id":"https:\/\/www.storagereview.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.storagereview.com\/news\/supermicro-ibm-storage-scale-nvidia-dynamo-kv-cache-platform-56x-ttft#primaryimage"},"image":{"@id":"https:\/\/www.storagereview.com\/news\/supermicro-ibm-storage-scale-nvidia-dynamo-kv-cache-platform-56x-ttft#primaryimage"},"thumbnailUrl":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2026\/11\/Supermicro-ASG-1115S-NE316R-Petascale-1U-E3S.jpg","datePublished":"2026-11-05T20:45:04+00:00","description":"Supermicro, IBM, and NVIDIA pair Petascale E3.S nodes, Storage Scale, and Dynamo in a KV cache tier claiming 56x faster TTFT at 130K tokens","breadcrumb":{"@id":"https:\/\/www.storagereview.com\/news\/supermicro-ibm-storage-scale-nvidia-dynamo-kv-cache-platform-56x-ttft#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.storagereview.com\/news\/supermicro-ibm-storage-scale-nvidia-dynamo-kv-cache-platform-56x-ttft"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.storagereview.com\/news\/supermicro-ibm-storage-scale-nvidia-dynamo-kv-cache-platform-56x-ttft#primaryimage","url":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2026\/11\/Supermicro-ASG-1115S-NE316R-Petascale-1U-E3S.jpg","contentUrl":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2026\/11\/Supermicro-ASG-1115S-NE316R-Petascale-1U-E3S.jpg","width":1200,"height":798,"caption":"Supermicro Petascale ASG-1115S-NE316R, the 1U 16-bay E3.S all-flash server used as the KV cache storage node in the IBM Storage Scale and NVIDIA Dynamo reference architecture"},{"@type":"BreadcrumbList","@id":"https:\/\/www.storagereview.com\/news\/supermicro-ibm-storage-scale-nvidia-dynamo-kv-cache-platform-56x-ttft#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.storagereview.com\/"},{"@type":"ListItem","position":2,"name":"News","item":"https:\/\/www.storagereview.com\/news"},{"@type":"ListItem","position":3,"name":"Supermicro KV Cache Platform With IBM Storage Scale and NVIDIA Dynamo Claims 56x Faster TTFT at 130K Tokens"}]},{"@type":"WebSite","@id":"https:\/\/www.storagereview.com\/#website","url":"https:\/\/www.storagereview.com\/","name":"StorageReview.com","description":"StorageReview.com is a leading provider of news and reviews throughout the entire IT stack - from the datacenter to the edge, and all points in between.","publisher":{"@id":"https:\/\/www.storagereview.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.storagereview.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.storagereview.com\/#organization","name":"StorageReview.com","url":"https:\/\/www.storagereview.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.storagereview.com\/#\/schema\/logo\/image\/","url":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2020\/02\/Storage-Reviews-2-2.png","contentUrl":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2020\/02\/Storage-Reviews-2-2.png","width":344,"height":61,"caption":"StorageReview.com"},"image":{"@id":"https:\/\/www.storagereview.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/storagereview","http:\/\/youtube.com\/user\/storagereview"]}]}
```