In 2021, Graid unveiled what they called the NVMe “RAID card,” though it was more than just that—it was a unique fusion of an NVIDIA GPU and Graid’s software. We’ve been big fans of SupremeRAID ever since, the performance is simply astounding and the integration is dead simple.
In 2021, Graid unveiled what they called the NVMe “RAID card,” though it was more than just that—it was a unique fusion of an NVIDIA GPU and Graid’s software. We’ve been big fans of SupremeRAID ever since, the performance is simply astounding.
A high-performance Gen5 Graid project that’s in process got us thinking about their solution. Being perpetually curious, we drifted off, thinking about some of our other favorite things in the lab. The problem is, we don’t have a free SupremeRAID card sitting about for whimsy. Never to let things like missing hardware stop our frivolous pursuits, we figured we could make something fun happen.
What if we combined the amazing Tyan AMD EPYC server with 26 of the capacious Solidigm 30.72TB SSDs and a skinny NVIDIA A2 GPU as the Graid card, to see just how much performance we can squeeze out of this exceedingly dense, low-cost platform?
The only problem really is the A2 selection. We chose the A2 due to the ease of installation and its “plug it in and go”, as it doesn’t need a power cable to work and fits in this server better than Graid’s standard card. That said, we should note that Graid doesn’t yet officially support the A2, their SupremeRAID SR-1010 runs on an A2000. But the A2 actually works better in this particular server and it was sitting idle, begging for something to do. Graid happily complied, delivering bits that we expected would happily engage with the A2.
What is Graid?
As a refresher, the Graid software is designed to maximize the potential of NVMe storage without being hindered by the traditional constraints of RAID configurations. By utilizing a GPU to handle RAID calculations, the software effectively frees up CPU resources, which can be better utilized for processing the actual workloads of databases and applications.
This approach not only boosts the performance but also simplifies the management of RAID systems, as it removes the need for complex hardware setups and allows for a more streamlined, software-centric management process. It’s a compelling solution for any enterprise looking to maximize their data center’s performance and efficiency without the overhead of traditional RAID cards or the resource drain of software RAID solutions.
Reflecting on our previous reviews of the Graid SupremeRAID, we were impressed by the technology’s ability to deliver almost 50GB/s in sequential reads and just over 10 million IOPS for 4K random reads for SR-1000 (Gen3), while hitting similar read speeds (though much higher write performance) with the SR-1010 (Gen4). This RAID solution not only shattered performance ceilings but also redefined the efficiency of high-speed storage. These figures were leaps ahead of the SW RAID5 configuration and easily best hardware RAID cards.
We expect more of the same and more when leveraging the TYAN server, NVIDIA A2 GPU, and the twenty-six Solidigm P5316 SSDs, but the switch to QLC does provide a bit of a wrinkle. While excellent in heavy read environments, the Solidigm drive does give back some performance as workloads shift to writes. Additionally, while we’ve tested the Graid card a lot, we’re in uncharted territory with the A2 and the massive capacity footprint these SSDs offer.
Graid SupremeRAID QLC Performance
As we indicated, we will be leveraging the TYAN Transport SX TS70AB8056 as the testbed to measure performance. We populated the TYAN server with twenty-six 30.72TB Solidigm P5336 SSDs, giving us a huge footprint to use in a RAID5 configuration. We are testing performance in both Graid HW RAID5 and mdadm software RAID5.
TYAN Transport SX TS70AB8056 Specifications:
- 1 x AMD EPYC 9684X CPU
- 8 x 64GB Kingston DDR5-4800 Registered ECC DRAM modules (KSM48R40BD4TMM-64HMR)
- 26 x Solidigm P5336 SSDs (30.72TB)
- SupremeRAID Driver: 1.5.0-659.g10e76f72.010
| Test | Graid HW RAID (4KB Stripe) |
SW RAID5 (mdadm) (4KB Chunk) |
| 1MB sequential read (192T/16Q) | 95.5GB/s, 33.7ms | 33.8GB/s, 95.4ms |
| 1MB sequential write (192T/16Q) | 12.7GB/s, 231.9ms | 1.7GB/s, 1,888ms |
| 64K random write (192T/16Q) | 1,894MB/s, 106.3ms | 2,051MB/s, 98.1ms |
| 4K random write (192T/32Q) | 255K IOPS, 12.1ms | 252k IOPS, 12.2ms |
| 4K random read (192T/32Q) |
19.1M IOPS, 0.32ms | 7.82M IOPS, 0.79ms |
Starting with 1MB sequential read bandwidth, the Graid configuration had a massive edge, measuring 95.5GB/s versus 33.8GB/s compared to the software RAID5, a healthy 3X performance win. Moving to sequential write, the substantial lead was the Graid config again. We measured 12.7GB/s from Graid compared to 1.7GB/s from the SW RAID5.
In our random 64K workload, we saw 1,894MB/s from Graid compared to 2,051MB/s with the SW RAID5. Here, the combined hardware solution just doesn’t offer much differentiation, clearly preferring the large block sequential workloads. This is likely a characteristic of the underlying QLC storage.
Moving down to the 4K random workload we saw parity with writes. The Graid configuration measured 255K IOPS compared to 252K IOPS on SW RAID5. In 4K random reads the story quickly turns back, the Graid more than doubled the performance with 19.1 million IOPS, as the software RAID5 configuration hit 7.82 million IOPS in comparison.
Conclusion
The performance results clearly favor Graid hardware RAID configurations for most scenarios, as it boasts awe-inspiring sequential read and write speeds that greatly surpass software RAID5. Worth noting, these numbers would beat a conventional hardware RAID card as well, which is limited to 28GB/s in a single PCIe slot, easily saturated by 4 SSDs.
Back to the H2H with software RAID, Graid’s sequential read throughput at 95.5GB/s is nearly three times that of software RAID5, making it an ideal solution for tasks requiring the rapid processing of large chunks of data. Moreover, its sequential write speed of 12.7GB/s is significantly better than the 1.7GB/s offered by software RAID5.
That said, software RAID5 is okay in specific use cases, such as random 64K write operations, where it slightly outperforms Graid. This suggests that software RAID5 can be more efficient for certain random-access workloads, but truth be told, this is more an artifact of the SSDs than Graid software or GPU.
Ultimately the Graid solution, even in our unsupported run, looks very strong with these high-capacity, low-cost SSDs from Solidigm. With massive data footprints becoming more common, enterprises will need a solution to easily wrangle their data. While software is one option and ASIC-based hardware RAID another, if organizations want to maximize the performance of their NVMe SSD investment, we highly recommend a Graid PoC first to see what you might be missing.
Engage with StorageReview
Newsletter | YouTube | Podcast iTunes/Spotify | Instagram | Twitter | TikTok | RSS Feed
