Historically, every storage read by a GPU has traversed the host CPU, which manages the file system, submits commands, and returns the data. This overhead is negligible for large sequential transfers but becomes a limiting factor when handling millions of 512-byte operations per second. At FMS 2026 in Santa Clara, NVIDIA announced new software and an industry initiative to eliminate this bottleneck.
The central announcement is cuFile, an API that lets GPUs read from and write to storage directly. NVIDIA is open-sourcing cuFile and its full supporting software stack, with the code hosted in the new XIO-SIG organization on GitHub. Founding maintainers include Google, Intel, Meta, and NVIDIA. In addition, NVIDIA formally launched Storage-Next, an initiative discussed publicly since at least GTC 2025. Storage-Next now includes over 40 storage and flash vendors, such as DDN, KIOXIA, and Micron, as well as controller manufacturers, cooling specialists, and standards organizations. These participants are collaboratively defining how storage devices should behave when a GPU, rather than a CPU, is the client.
The Problem: Host-Mediated I/O
GPU systems have traditionally operated under a host-centric I/O model. The CPU schedules GPU tasks, stages data, and initiates all transfers between the GPU and the network or storage. Each operation follows a sequence: the GPU kernel completes, control returns to the CPU, the CPU sets up the transfer, and the next kernel launches. The per-operation cost, mainly kernel-launch and CPU-GPU synchronization overhead, is measured in microseconds. While negligible for bulk transfers, this overhead becomes significant for fine-grained operations.
For GPU-to-GPU communication, NVIDIA eliminated host involvement through NVSHMEM, which allows device code to initiate transfers directly and overlap them with computation. The impact is substantial: when GROMACS transitioned its halo-exchange communication from MPI to NVSHMEM, communication was removed from the critical path and became fully overlapped with computation.
Storage kept the host-mediated model. A read from an NVMe drive still returns control to the CPU, runs the file system, and waits for completion. Training tolerates this because its reads are large and infrequent. Inference does not. Serving KV cache entries, embeddings, and vector-search results generates millions of small random reads per second, with the CPU in the path of every one.
GPU-Initiated I/O and SCADA
SCADA (scaled, accelerated data access) applies the NVSHMEM model to storage, enabling the GPU to initiate and manage its own storage I/O.
The underlying research is BaM, or Big Accelerator Memory, published at ASPLOS 2023 by NVIDIA, IBM, the University of Illinois, and the University at Buffalo. BaM demonstrated a GPU managing NVMe drives directly, coalescing and caching the small requests generated across its threads to sustain drive-saturating queue depths without host involvement. On identical hardware, BaM ran data analytics workloads 5.3x faster than CPU-initiated access and executed graph workloads from flash at up to 21.7x lower hardware cost than storing the same data in host memory.
SCADA is the productized form. A runtime on the GPU sits between the kernel’s threads and storage. It coalesces scattered small requests into reads served from an on-GPU cache or issued to drives, which can be local NVMe or a remote storage server.
Direct device access must be contained since raw access to a shared drive could let one application read or corrupt another’s data. NVIDIA’s design separates privilege levels. Performance-critical parts of an application run unprivileged, outside the trusted computing base. A privileged component configures protected access between each application and only its approved storage at setup, using standard Linux security mechanisms. NVIDIA cites this split and Linux-based interoperability as the reason the stack is released as open source rather than a closed CUDA component.
How SCADA Differs From GPUDirect Storage
Every storage operation has a data path (the bytes moved) and a control path (request construction, submission, and completion handling). GPUDirect Storage (GDS), introduced in 2019 and shipping since 2021, removed the CPU from the data path: through cuFile, data moves by DMA between the drive and GPU memory with no bounce buffer in host RAM. NVIDIA’s launch materials put the CPU-path ceiling on a DGX-2 at 50GB/s, with GDS able to combine paths for an upper limit approaching 200GB/s, and DDN, VAST, WEKA, Pure Storage, and the other major AI storage vendors certified against it. The control path stayed on the CPU: host software still decides what to fetch and issues every request, and the GPU is the DMA target rather than the initiator.
At large transfer sizes, this split is fine. A 1MB read amortizes the control-path cost to nothing. At 512 bytes, the ratio inverts. The fixed per-request cost dominates, and the CPU saturates well before the drives do. SCADA moves the control path onto the GPU, which absorbs per-operation latency the same way it absorbs memory latency, by keeping hundreds of thousands of operations in flight.
The two coexist: cuFile for bulk transfers, SCADA for high volumes of small random reads.
Storage-Next and the 512-Byte Problem
SCADA covers the GPU side. Storage-Next addresses the drive side, which is currently optimized for a different workload.
Enterprise SSDs have been tuned for 4KB random reads for a decade, matching database and virtualization access patterns. Most controllers do roughly the same work to serve a 512-byte read as a 4KB one, so sub-4K requests run at 4K cost. Inference access patterns are smaller: embeddings run a few hundred bytes, and KV cache blocks are well under a kilobyte. Serving them from 4K-tuned drives imposes 8x read amplification, wasting bandwidth and controller cache. At tens of terabytes of small objects, that amplification determines whether flash is viable as a memory tier.
Storage-Next is the effort to retool drives, controllers, and systems around the smaller unit. It surfaced publicly at GTC 2025, already underway, with a stated goal of maximizing 512-byte IOPS per GPU under power and tail-latency constraints. At FMS, NVIDIA attached a membership number, more than 40 storage and flash vendors, and designated SCADA as the framework to build against.
The reference result is Micron’s SC25 demonstration of 230 million 512-byte random-read IOPS from a single server under the SCADA programming model: 44 Micron 9650 PCIe Gen6 SSDs behind three Broadcom PEX90000 Gen6 switches, driven by three H100s in an H3 Platform Falcon 6048 chassis. The figure works out to roughly 95% of the 44 drives’ combined 5.5-million-IOPS ratings.
The composition of that number matters. 230 million 512-byte reads equal roughly 118GB/s of payload, which four or five of those drives could deliver sequentially. The achievement is the operation count. Sustaining it through host software would consume dozens of CPU cores on submission and completion handling alone, yet the demo’s single CPU was essentially idle. Distributed across a GPU’s hundred-thousand-plus threads, the load is a few thousand operations per thread per second.
SSD capability is now the gating factor. Micron’s 9650, the first PCIe Gen6 SSD, is rated at 28GB/s sequential and a record 5.5 million random-read IOPS. Micron reports linear scaling from 1 to 44 drives in the demo. Kioxia’s GP Series, an XL-Flash SSD built for 512-byte access, is being developed under Storage-Next. NVIDIA’s Storage-Next roadmap calls for PCIe Gen7 SSDs sustaining 100 million IOPS each, a target controller vendors, including Marvell, are now designing toward. Reaching those numbers requires controller redesign, error correction sized for small payloads, and standards defining GPU-native drive behavior.
Mapping to Training and Inference
The two paths correspond to two access patterns. Training is bandwidth-bound. It streams large sequential shards and periodically writes multi-terabyte checkpoints during which GPUs idle. CPU overhead is negligible at those transfer sizes, so GDS and the newly open-sourced cuFile stack remain the correct interface, measured in GB/s per GPU and checkpoint duration.
Inference is IOPS-bound. The KV cache, the attention state for every token already processed in a conversation, grows with context length. Agentic deployments run thousands of concurrent conversations. It quickly exceeds GPU memory, and recomputing evicted entries costs more GPU time than reading them back. The standard design is a tiered cache spilling from GPU memory to system memory to flash, refilled through high volumes of small random reads. Vector search and embedding lookups produce the same pattern. This is the access shape SCADA targets, and the reason NVIDIA quotes 512-byte IOPS rather than bandwidth. Serving KV cache from flash rather than memory raises the context and concurrent-user count each GPU supports, which sets the per-user serving cost.
The Hardware: Vera BlueField-4 STX and CMX
The platform layer is NVIDIA Vera BlueField-4 STX, the rack-scale storage architecture introduced at GTC 2026 and extended this week. STX pairs the Vera Rubin platform with BlueField-4 storage processors, each combining Vera Arm cores with 800Gb/s of integrated networking and roughly 6x the compute of BlueField-3.
At FMS, NVIDIA published a benchmark for those cores. A two-stage compression-and-encryption pipeline, representative of the inline data services storage systems that apply to all traffic, ran at up to 3.21x the throughput of an x86 CPU on Vera. The implied claim is that encryption, compression, and verification, which currently consume x86 cores in storage controllers, can move to Vera silicon at lower power. DOCA, NVIDIA’s software framework for BlueField, enforces security policy in the data path. CMX Context Memory Storage, built on STX, pools flash behind BlueField-4 as a shared KV-cache tier for long-context inference. Partner systems from DDN, Dell, HPE, IBM, VAST Data, WEKA, and others are due in the second half of 2026.
Closing Thoughts
The strategic structure is clear: NVIDIA is publishing the interface GPUs will use to reach storage, open-sourcing the implementation, and organizing 40-plus vendors to standardize the hardware behavior underneath it. Opening cuFile departs from NVIDIA’s practice of keeping this layer inside CUDA. But a GPU-initiated storage interface only pays off if drive, controller, and array vendors build to it. Intel joining as a maintainer is a notable signal. A leading supplier of the x86 silicon in current storage systems is backing software designed to remove that silicon from the I/O path.
Micron’s demonstration establishes feasibility. Three GPUs drove 44 Gen6 drives to roughly 95% of their combined rated limit at a block size the industry has treated as an afterthought. What remains is execution: the code landing on GitHub, SCADA maturing from framework to supported CUDA component, and vendors shipping small-read-optimized SSDs in volume. The figures above are NVIDIA’s and its partners’, not independent measurements. We will test them when STX systems reach the lab in the second half of the year.
Amazon