The Apex Storage X21 brings top-tier performance and capacity to enterprise and prosumer use cases with 21 Gen4 NVMe M.2 SSDs on a double-width PCIe add-in card (FHFL). In total, the X21 can provide 168TB of storage per card (8TB SSDs) and cranks out up to 31GB/s read speeds and over 10 million IOPS.
The Apex Storage X21 brings top-tier performance and capacity to enterprise and prosumer use cases with 21 Gen4 NVMe M.2 SSDs on a double-width PCIe add-in card (FHFL). In total, the X21 can provide 168TB of storage per card (8TB SSDs) and cranks out up to 31GB/s read speeds and over 10 million IOPS.
Apex Storage X21 Overview
The X21 features plug-and-play compatibility and supports UEFI and Secure Boot. It offers enterprise-grade reliability with advanced ECC and data protection, as well as advanced error recovery. Its low overhead architecture allows for no host CPU or DRAM offload, and it also supports software or OS RAID for data protection. Drivers are pre-compiled in Windows and Linux, so deployment is simple (but faster on Linux), and the card works with VMware ESXi.
Driving this feature set and the overall performance is the Microchip 100 lane Gen4 PM40100 Fanout PCIe Switch. The chip has a TDP of 25 watts, meaning that the controller will stay cool with light airflow thanks to the included heatsink. That said, the SSDs may need a little extra airflow, which will depend on the drives and the enclosure the card is going in. It should also be noted that while space is tight, there is room for a 4mm heatsink to be added to the SSDs if needed. Apex also has a little design wiggle room. For custom applications, they can adjust the width of the card via the interconnect PCB.
The materials of the card also make a difference. Apex lists the Storage X21 as made from Thunderclad 3+ material. What is it? Well Thunderclad or TU-933+, which, at the time of specification, is the highest quality PCB material available. While lower-loss materials have emerged during the card’s development, they claim it has helped the X21 continue to provide exceptional performance. While we don’t have a test for such a thing, it’s an interesting side note that we’re not usually made aware of.
Apex Storage is targeting a few obvious use cases with the card; most clearly, the X21 fits the needs of data scientists and creative pros who want the fastest possible local storage. Apex points out they feel the X21 is appropriate for a number of enterprise use cases, like HCI. Unfortunately, enterprise SSD vendors have pulled back on M.2 development, opting for EDSFF drives instead. While there may be quite a few spots in the data center where the X21 fits, we’re thinking at the moment it’s a little more of a point solution.
At the same time, let’s not dismiss the card’s capabilities. The X21 absolutely flies, aggregating the performance of 21 SSDs. As we’ll see in our performance results below, Apex’s spec sheet claims are reasonable. And we’re not even using the best drives available. While the 980 Pros used in this review are certainly capable, for the absolute best performance, there’s even more potential on the table. And should a single card not meet your performance goals, Apex has validated that four cards with 84 drives works just fine. In the right system, a user could conceivably aggregate 8 of these cards in a server with a whopping 168 SSDs.
Our emphasis is clearly on the storage capabilities this time around, but the X21 is more than just a one-trick pony. The M.2 slot supports a number of other devices. For instance, you could use wireless cards with M.2 A+E converter boards to set each wireless card to a specific wireless channel for spectrum monitoring. There’s even an AI angle here. A company called Axelera makes an M.2 AI Edge accelerator module that could be used by the Apex card at some point as well.
The X21 sells for $2800 as a single unit, with volume pricing and custom orders available.
Apex Storage X21 Specifications
| Raw Capacity | 168TB w/ 8TB M.2 NVME SSDs (336TB w/ future 16TB M.2 SSDs) |
| Drive Interface | M.2 Pci-e Gen 4 |
| Read Bandwidth | 30.5 GBps (107+ GBps Multicard) |
| Write Bandwidth | 28.5 GBps (80+ GBps Multicard) |
| Read/Write IOPS | 10 Million + |
| Read Access Latency | 79us avg. |
| Write Access Latency | 52us avg. |
| Host Protocol | Pci-e Gen 4 16x |
| M.2 Protocol | Gen 4 NVME v2.0/v1.4c |
| Warranty | Lifetime |
| Weight | 19 oz |
| Power | Active: ~95 Watt Typical 2 x PCI-e 6 pin 12v |
| Air Flow | Min 400 LFM |
| Humidity | 5% to 95% (non-condensing) |
| Supported OS | Windows, Windows Server, Linux |
Apex Storage X21 Performance
Despite the Apex Storage X21’s remarkable capabilities, it’s important to note the challenges we faced regarding system compatibility. To properly boot and benchmark the card, we found ourselves resorting to a consumer platform, as the X21 encountered difficulties when integrated with enterprise servers and workstations in our lab.
This limitation may restrict the card’s adaptability for some users and could potentially hinder its seamless integration into a variety of configurations. It’s crucial for prospective buyers to thoroughly assess their system compatibility before investing in the X21 to ensure a smooth and efficient deployment of this powerful storage solution.
To leverage this card, the bare minimum requirement is a dual-slot opening, a x16 PCIe slot, and a 6-pin power feed. For a consumer workstation, this is the same requirement as a traditional full-size GPU, and most desktops will have the space and power availability.
On the server side, this gets more complicated. Traditional 1U and 2U servers are built to take advantage of more normal PCIe edge cards, primarily the common HHHL (half-height, half-length) card. Almost all of these draw power from the PCIe slot, so many servers don’t ship with add-on power connectors. For both size and power, you’d want to leverage a GPU-ready server with the appropriate size bays, airflow, and, most importantly, power availability.
For this review, we leveraged an AMD-based MSI X670E Motherboard with an AMD Ryzen 9 7950X 4.5GHz, 16-core CPU. This platform included 64GB of DDR5 memory and, for the duration of the review, leveraged integrated graphics only. We had tried using our Lenovo and Dell AMD ThreadRipper platforms but ran into issues getting the platforms to post. Our Intel-based consumer platform connected to the card in x8 instead of x16 mode, and the BIOS didn’t include an option to set the PCIe width manually. The processor inside our rig also started to choke in the multi-million IOPS tests.
Vdbench Performance Tests
To kick the tires on the Apex Storage X21 with 21 Samsung 980 Pro 1TB SSDs, we used a Debian OS test environment with our Vdbench tests, as well as some Fio workloads. The Vdbench workloads provide some comparison to enterprise devices we’ve reviewed over the years, while Fio supported the newer io_uring I/O protocol, which enables lower-latency, high-throughput testing.
When it comes to benchmarking storage devices, application testing is best, and synthetic testing comes in second place. While not a perfect representation of actual workloads, synthetic tests do help to baseline storage devices with a repeatability factor that makes it easy to do apples-to-apples comparisons between competing solutions. These workloads offer a range of different testing profiles ranging from “four corners” tests and common database transfer size tests to trace captures from different VDI environments.
All of these tests leverage the common vdBench workload generator, with a scripting engine to automate and capture results over a large compute testing cluster. This allows us to repeat the same workloads across a wide range of storage devices, including flash arrays and individual storage devices. Our testing process for these benchmarks fills the entire drive surface with data and then partitions a drive section equal to 5% of the drive capacity to simulate how the drive might respond to application workloads. This is different from full entropy tests, which use 100% of the drive and takes them into a steady state. As a result, these figures will reflect higher-sustained write speeds.
Profiles:
- 4K Random Read: 100% Read, 128 threads, 0-120% iorate
- 4K Random Write: 100% Write, 128 threads, 0-120% iorate
- 64K Sequential Read: 100% Read, 32 threads, 0-120% iorate
- 64K Sequential Write: 100% Write, 16 threads, 0-120% iorate
- Synthetic Database: SQL
- VDI Full Clone Traces
While the Dapustor R5100 and Apex Storage X21 don’t quite compete head to head, we picked this drive as it was one of the fastest U.2 enterprise SSDs we’ve tested recently. While the Dapustor R5100 topped out at nearly 1.8M IOPS, the Apex Storage X21 screamed past it, with much lower latency, coming in at just under 3M IOPS in this test.
Next we look at 4K random write performance, where the Dapustore offered around 700k IOPS, while the X21 came in with 1.94M IOPS with the 21 Samsung 980 Pro SSDs.
While 4K performance had a dramatic difference in the scale of performance, that’s nothing compared to what you see looking at the drive bandwidth. The Apex Storage X21, besides a GPU, is one of the few devices that takes full advantage of every drop of PCIe potential, and it completely saturates the x16 Gen4 slot. In our 64K sequential workload, the X21 came in with an incredible 27.2GB/s, whereas the Dapustor measured 7.2GB/s.
64K sequential write performance was equally as impressive, with the X21 coming in with 23.3GB/s while the Dapustor R5100 topped out at 2.8GB/s.
Next, we look at our SQL Workload, in which the X21 offered a peak throughput of 3.46M IOPS or 27.1GB/s.
In our SQL 90-10 workload, the Apex Storage X21 measured 3.27M IOPS or 25.6GB/s.
In our SQL 80-20 workload with a higher write mixture, the X21 measured a peak throughput of 3.13M IOPS or 24.5GB/s.
Finally, we look at VDI performance with our full-clone traces. In our VDI Initial Login test, the X21 came in with an impressive 1.01M IOPS or 19.2GB/s.
In our VDI Boot workload, the Apex Storage X21 measured 1.99M IOPS or 27.5GB/s, completely saturating the PCIe Gen4 slot.
In our last Vdbench workload covering VDI Monday Login, the X21 measured 919k IOPS or 20GB/s at its peak.
While Vdbench covers one aspect of the drive’s synthetic performance, we also included Fio results in this review as it can leverage the newer and more efficient io-uring I/O protocol. Here our focus is showing the absolute peak I/O and bandwidth possible through the card. Before each of these tests was performed, each SSD was formatted and then completely filled.
| Workload | Throughput (IOPS) | Bandwidth (GB/s) |
|---|---|---|
| 512-byte Random Read | 14.6M IOPS | 7.477GB/s |
| 512-byte Random Write | 3.421M IOPS | 1.751GB/s |
| 4K Random Read | 6.832M IOPS | 28GB/s |
| 4K Random Write | 5.274M IOPS | 21.6GB/s |
| 8K Random Read | 3.454M IOPS | 28.3GB/s |
| 8K Random Write | 3.499M IOPS | 28.7GB/s |
One interesting stat we wanted to mention is regarding the 512-byte random read performance. Before the SSDs were filled, they could push 23M IOPS at 11.8GB/s. While this was different from the true performance of these SSDs before getting completely filled with data, it does indicate the Apex Storage X21 is capable of passing far more traffic with faster SSDs.
We’ve been told by the company that performance greater than 40M IOPS is possible. The main limit this card will hit is the PCIe Gen4 x16 slot cap, which will top out at around 28GB/s or so.
Apex Storage X21 Thermals
When we first saw the Apex Storage X21 hit the news wire, our first thought was how fast will this drive burst into flames. With how much power is flowing through each of the 21 SSDs this card is capable of holding, thermal performance is no joke.
The Samsung 980 Pro SSD at peak load is capable of drawing almost 8.5W of power, which, with all of them combined, would be using nearly 180W on this card. All that power will turn to heat, and you need to shed it quickly before the drives or card gets throttled or damaged.
To safely test this card in our test rig, we leveraged a massive AC-powered fan positioned next to the motherboard for ample airflow. Apex lists the airflow requirements at 400LFM, which is well outside the realm of a normal desktop and even most workstations. That type of airflow is what you’d commonly find in a server while it is operating at near full tilt in terms of fan speed.
We monitored the individual drive temperatures reported through SMART data during our full drive fill. This put each drive into its worst-case scenario of full write performance over an extended period of time. The temperatures were polled towards the end of this activity.
| Drive | Temperature |
|---|---|
| SSD1 | 51°C |
| SSD2 | 44°C |
| SSD3 | 59°C |
| SSD4 | 44°C |
| SSD5 | 63°C |
| SSD6 | 46°C |
| SSD7 | 57°C |
| SSD8 | 46°C |
| SSD9 | 45°C |
| SSD10 | 59°C |
| SSD11 | 45°C |
| SSD12 | 61°C |
| SSD13 | 46°C |
| SSD14 | 56°C |
| SSD15 | 49°C |
| SSD16 | 52°C |
| SSD17 | 34°C |
| SSD18 | 41°C |
| SSD19 | 36°C |
| SSD20 | 35°C |
| SSD21 | 42°C |
The safe operating range of the Samsung 980 Pro is listed at 0-70C, so at their peak load, the hottest drives did stay under that figure. Making sure the cooling is in check is an absolute requirement before putting one of these into service.
Final Thoughts
The Apex Storage X21 is a truly remarkable and groundbreaking piece of hardware that brings high performance and vast storage capacity to enterprise and prosumer environments. However, it is important to be aware of its limitations, such as the significant airflow requirements and heat generation, as it draws power comparable to a GPU without built-in cooling.
Despite the installation and cooling challenges, with a proper setup and cooling solutions, the X21 has the potential to unlock unparalleled performance for various applications. The X21’s impressive storage capabilities could play a crucial role in powering data-intensive applications that are popular in data science and creative professional workloads.
It’s always a fun surprise when new devices come into our lab and completely breaks or saturates a system. The Apex Storage X21 did just that with its incredible performance capabilities. We could pull 28.3GB/s off the card in an 8K random read operation or push 28.7GB/s to it in an 8K random write workload. For small-block write in our 4K Fio test, this card topped out at 6.8M IOPS read and 5.3M IOPS write. Moving to a smaller blocksize, we started to hit the limits of the Samsung 980 Pro SSDs, which could push 23M IOPS in a 512-byte random read operation factory fresh but dropped down to 14.6M IOPS once the drives were filled. To hit these figures out of a single slot is incredible.
The X21 offers unparalleled performance to the point that this one component could be a productivity game-changer that a certain set of professionals are looking for. Our review unit is $4500 all in with the SSDs, which isn’t that bad, considering the value it will provide to applications that need this level of performance. Again, the X21 needs a pretty prescriptive platform to work, but if you’re willing to make that effort, you will not be disappointed with what this card brings to the table. In fact, you’ll probably be downright giddy.
Engage with StorageReview
Newsletter | YouTube | Podcast iTunes/Spotify | Instagram | Twitter | TikTok | RSS Feed
