We like talking about flash because, well, it’s fast and makes applications smile thanks to low latency. But the reality is, hard drives still dominate the storage landscape whether you’re a hyperscaler or a small business — our readers certainly don’t need the hyperbolic rant about the speed of data growth. To help address these rapidly growing data needs, be it backup, media files, or unstructured data, Supermicro has many solutions. In this review, we take a look at Supermicro’s latest mainstream storage server that features 36 3.5″ hard drive bays and Intel’s latest 3rd Gen Scalable CPU.
We like talking about flash because, well, it’s fast and makes applications smile thanks to low latency. But the reality is, hard drives still dominate the storage landscape whether you’re a hyperscaler or a small business — our readers certainly don’t need the hyperbolic rant about the speed of data growth. To help address these rapidly growing data needs, be it backup, media files, or unstructured data, Supermicro has many solutions. In this review, we take a look at Supermicro’s latest mainstream storage server that features 36 3.5″ hard drive bays and Intel’s latest 3rd Gen Scalable CPU.
Supermicro Storage SuperServer SSG-540P-E1CTR36H
Supermicro offers a wide variety of storage servers, in additional to pure JBODs that are similar looking. Of course, the top of the line are NVMe servers. There’s also a line of destiny-optimized offerings in the top-loading storage server family. These are more designed for hyperscalers and other use cases where terabytes per rack U is the primary requirement. For the rest of the world, Supermicro offers what it calls enterprise-optimized storage servers. This family covers your commonly found 1U-4U options with externally accessed drive bays.
In the case of this review, the SSG-540P-E1CTR36H is a double-sided version of the SuperServer. It’s 4U and supports a total of 36 3.5″ drive bays for storage. There are 24 in the front and another 12 in the rear. Onboard SATA (6Gbps) is supported via the C621A Chipset which offers RAID 0, 1, 5, and 10 support. SAS connectivity for the 3.5″ bays (12Gbps) is handled via a Broadcom 3908 with RAID 0, 1, 5, 6, 10, 50, and 60 support. It should be noted that there is another flavor of this server, the SSG-540P-E1CTR36L. The “L” model differs just in the SAS card, opting for the Broadcom 3808 (IT mode) instead. Either version can be equipped with two hot-swap 2.5″ NVMe/SATA bays on the rear.
Internally, the Supermicro Storage SuperServer includes a single NVMe/SATA M.2 bay for boot. Should you prefer, there are also two SuperDOM (Disk on Module) ports with built-in power on the board. For those looking to be more adventurous with the internal storage, there is support for two internal fixed bay modules with support for up to 4x 2.5″ drives in total.
This system is a single-socket server, supporting Intel Ice Lake CPUs up to 270W TDP. The board also offers eight DDR4-3200 DIMM slots that do support Intel PMem, though that would be overkill for most in this particular system. The underlying motherboard is the Supermicro Super X12SPI-TF.
Looking at expansion, Supermicro includes two PCI-E 4.0 x16 (LP) slots and two PCI-E 4.0 x8 (LP) slots. This gives the end-user quite a bit of flexibility via add-in cards. Supermicro has equipped the server with twin Intel Ethernet Controller X550 10GbE RJ45 NICs. There’s also a GbE port for IPMI server management. Visible in the above photo and on the rear-view of the chassis is an add-on card for JBOD expansion, which ties in directly to the SAS expander on this server. This allows for easy and affordable expansion of the storage pool on the server since it ties in directly to the primary RAID card.
Supermicro Storage SuperServer SSG-540P-E1CTR36H Specifications
| Product SKUs | ||
| SSG-540P-E1CTR36H |
|
|
| Motherboard | ||
| Super X12SPI-TF | ||
| Processor | ||
| CPU |
|
|
| System Memory | ||
| Memory |
|
|
| Memory Voltage |
|
|
| Error Detection |
|
|
| On-Board Devices | ||
| Onboard SATA or Optional Rear Bay |
|
|
| 36-bay SAS connectivity |
|
|
| Chipset |
|
|
| Network Connectivity |
|
|
| IPMI |
|
|
| Input / Output | ||
| LAN |
|
|
| USB |
|
|
| Video |
|
|
| Serial Port |
|
|
| DOM |
|
|
| System BIOS | ||
| BIOS Type |
|
|
| Management | ||
| Software |
|
|
| Power Configurations |
|
|
| PC Health Monitoring | ||
| CPU |
|
|
| FAN |
|
|
| Temperature |
|
|
| Chassis | ||
| Form Factor |
|
|
| Model |
|
|
| Dimensions and Weight | ||
| Height |
|
|
| Width |
|
|
| Depth |
|
|
| Package |
|
|
| Weight |
|
|
| Available Color |
|
|
| Front Panel | ||
| Buttons |
|
|
| LEDs |
|
|
| Expansion Slots | ||
| PCI-Express (PCI-E) |
|
|
| Drive Bays / Storage | ||
| Hot-swap |
|
|
| M.2 |
|
|
| Peripheral Bays |
|
|
| System Cooling | ||
| Fans |
|
|
| Air Shroud |
|
|
| Power Supply | ||
| AC 1200W, Titanium Level, Redundancy,PMBUS 1.2, +12V/+5Vsb, 360x76x40 mm,HF,RoHS/REACH | ||
| Dimension
(W x H x L) |
|
|
| Output Type |
|
|
| Certification | Titanium Level | |
| Operating Environment | ||
| Environmental Spec. |
|
|
Supermicro Storage SuperServer Configuration
For this review, we secured 32 Western Digital Ultrastar DC HC520 12TB hard drives. 32 may seem an odd amount for a 36-bay storage server. Either I asked for four short of our count or WD said we could only have 32. Either way, we’re still very well stocked on storage, the 12TB drives give us 384TB of raw storage with four drive bays for expansion or hot spares.
If we were going for a density hero number, of course, WD’s 20TB HDDs would be preferable. But the reality is that in the enterprise and certainly with most SMBs that have a large data footprint, economics enters the conversation sooner than later. The 12TB HC520 can be found for under $400 in retail, and that’s not even a volume price. The drives are still in production and have been one of WD’s strongest workhorse HDDs in this capacity point. Our review units have a November 2021 build date.
For its part, the HC520 is available in either a SAS or SATA part with capacities ranging up to 12TB. The drives are helium sealed and feature a 7200 RPM spindle speed. Each drive is rated to deliver a maximum of 255MB/s, meaning our Supermicro Storage SuperServer with 32 drives can deliver a best-case 8,000MB/s which will easily saturate its twin 10GbE ports in some workloads. The full WD HC520 spec sheet can be found here.
With the capacity storage sorted, we installed a Supermicro SATADOM for boot duties. While many will prefer the M.2 option for boot, the and user has options. To help us aggregate the storage and share it out to the network, we installed TrueNAS CORE. With the specifications available, even in this single-socket configuration, the argument could be made that internal benchmarks would be appropriate. We simply opted to leverage shared storage via TrueNAS for this review.
Supermicro Storage SuperServer Performance
To show just how easy it is to get this platform rolling, we deployed TrueNAS CORE on this Supermicro Storage SuperServer, leveraging the onboard Intel dual-port 10GbE interface for connectivity. Leveraging the 32-supplied WD HDDs, we opted for RAIDZ3, giving us protection from three drive failures.
Using the default filesystem configuration, we had LZ4 compression enabled with deduplication turned off. We then created four shared folders for testing SMB performance and four 25GB iSCSI LUNs for block storage performance.
Enterprise Synthetic Workload Analysis
Our enterprise hard drive benchmark process preconditions each drive-set into steady-state with the same workload the device will be tested with under a heavy load of 16 threads, with an outstanding queue of 16 per thread. The device is then tested in set intervals in multiple thread/queue depth profiles to show performance under light and heavy usage. Since hard drives reach their rated performance level very quickly, we only graph out the main sections of each test.
Preconditioning and Primary Steady-State Tests:
- Throughput (Read+Write IOPS Aggregate)
- Average Latency (Read+Write Latency Averaged Together)
- Max Latency (Peak Read or Write Latency)
- Latency Standard Deviation (Read+Write Standard Deviation Averaged Together)
Our Enterprise Synthetic Workload Analysis includes four profiles based on real-world tasks. These profiles have been developed to make it easier to compare to our past benchmarks, as well as widely-published values such as max 4K read and write speed and 8K 70/30, which is commonly used for enterprise drives.
- 4K
- 100% Read or 100% Write
- 100% 4K
- 8K 70/30
- 70% Read, 30% Write
- 100% 8K
- 128K (Sequential)
- 100% Read or 100% Write
- 100% 128K
Looking at our throughput test which measures 4K random performance, the Supermicro SuperServer (populated with WD HC520 12TB HDDs) showed iSCSI performance of 309,184 IOPS read and 15,560 IOPS write. In SMB, the Supermicro SuperServer hit 107,507 IOPS read and 15,228 IOPS write.
Next is 4K average latency. Here, the Supermicro SuperServer hit 16.45ms latencies of write and 0.83ms read in the iSCSI configuration while SMB recorded 16.81ms write and 2.38ms read.
With 4K max latency, the Supermicro SuperServer showed 28,810ms read and 171.4ms write in iSCSI while hitting 118.1ms read and 94ms write in SMB.
In standard deviation, the Supermicro SuperServer showed reads and writes of 18.77ms and 13.68 ms in iSCSI, respectively, while SMB showed 1.82ms read and 10.67ms write in SMB.
The next benchmark tests the drives under 100% read/write activity, but this time at 8K sequential throughput. In iSCSI, the Supermicro SuperServer hit 252,407 IOPS read and 280,627 IOPS write, while SMB saw 108,928 IOPS read and 102,941 IOPS write.
Our next test shifts focus from a pure 8K sequential 100% read/write scenario to a mixed 8K 70/30 workload, which will demonstrate how performance scales in a setting from 2T/2Q up to 16T/16Q. In SMB, the Supermicro SuperServer started at 16,538 IOPS and ended at 42,407 IOPS in the terminal queue depths. In iSCSI, we saw a range of 16,971 IOPS to 53,071 IOPS.
With average latency at 8K 70/30, the Supermicro SuperServer showed a range of 0.23ms through 4.82ms in iSCSI, while SMB showed a range of 0.23ms through 6.03ms.
For max latency, the Supermicro SuperServer posted a range of 12.67ms to 244.42ms in iSCSI, while SMB showed 10.03ms through 42.1ms in the terminal queue depths.
The standard deviation latency results, the Supermicro SuperServer had a range of 0.08ms to 3.86ms in iSCSI while SMB recorded 0.07ms through 1.84ms.
Our last test is the 128K benchmark, which is a large-block sequential test that shows the highest sequential transfer speed. Here, the Supermicro SuperServer posted 2.32GB/s read and 2.31GB/s write in iSCSI, while SMB had 2.31GB/s read and 2.31GB/s write.
Conclusion
The Supermicro Storage SuperServer is a 36-bay mainstream storage server powered by Intel’s latest 3rd Gen Scalable CPU and is a great solution to help organizations with their rapidly growing data needs, such as backup, media files, or unstructured data.
For performance, we tested the Supermicro SuperServer using both iSCSI and SMB connectivity, populating the server with the 32 Western Digital Ultrastar DC HC520 12TB hard drives. We installed TrueNAS CORE to share the storage over the network, the drives in RAIDZ3 for resiliency. We used the native twin Intel 10GbE ports on the Supermicro box.
With our FIO workloads measuring RAID3Z performance across 32 HDDs, the Supermicro SuperServer showed 4K random performance of 309,184 IOPS read and 15,560 IOPS write in iSCSI while hitting 107,507 IOPS read and 15,228 IOPS write in SMB. Looking at workloads under 100% read/write activity 8K sequential throughput, the Supermicro SuperServer hit 252,407 IOPS read and 280,627 IOPS write (iSCSI) and 108,928 IOPS read and 102,941 IOPS write (SMB). In our mixed 8K 70/30 workload, which will demonstrate how performance scales in a setting from 2T/2Q up to 16T/16Q, the SuperServer started at 16,538 IOPS and ended at 42,407 IOPS in the terminal queue depths in SMB, while seeing a range of 16,971 IOPS to 53,071 IOPS in iSCSI.
Finally, in our large-block sequential 128K benchmark, the Supermicro SuperServer posted 2.32GB/s read and 2.31GB/s write in iSCSI and 2.31GB/s read and 2.31GB/s write in SMB. At this point we were saturating both onboard 10GbE NICs, showing that this server has much more to give. If your use case is dependent on large block sequential workloads, it may be worth investing in a faster add-in card. This server has plenty of room for expansion.
Storage servers of this nature have been fundamental to supporting a wide range of use cases over the years, supporting the explosion of data in the enterprise. Supermicro has done well with this server, offering a progressive update thanks to the new 3rd Gen Intel Scalable CPU. Even though this rig is designed for optimized capacity, Supermicro does offer support for 2.5″ NVMe bays in the back and the board supports Intel PMem, in the event you need this thing to really scream. For everyone else this particular combo of Supermicro gear, affordable 12TB WD hard drives and TrueNAS CORE layered on top, will be more than enough.
Engage with StorageReview
Newsletter | YouTube | Podcast iTunes/Spotify | Instagram | Twitter | Facebook | TikTok | RSS Feed
