June 19th, 2015 by Lyle Smith
Supermicro SuperServer 1028U-TNR4T+ Review
The Supermicro SuperServer 1028U-TNR4T+ is a high-end server that is comprised of the SC119UAC(2)-R750 1U server chassis featuring and X10DRU-i+ dual processor serverboard subsystems, the latter which is based on the Intel PCH C612 chipset. Like the previously reviewed SuperServer 1027R-WC1NRT, the 1028U provides enterprises with a ton of versatility for heavy application workloads with its equipped 10 hot-swappable 2.5" SAS/SATA drive bays.
The most notable feature (and biggest draw) of the Supermicro SuperServer is its two NVM Express ports, allowing the 1028U to boast high-speed, low-latency connections directly from the CPU to NVMe SSDs, which results in a significant improvement over SATA or SAS SSDs and a reduction in storage device latency. This is made possible due to the direct PCI-E interface between the CPU and the NVMe SSDs. There are very few other mainstream server vendors that offer this unique compatibility with NVMe drives, another being Dell with select PowerEdge servers.
Under the hood of the 1028U lies 24 DIMM slots, which support up to 1536 GB of LRDIMM or up to 768 GB of RDIMM DDR4-2133/1866/1600/1333 SDRAM. A SATA controller has been integrated into the chipset, providing users with ten-port SATA subsystem, with six of them supported by the Intel PCH and four supported by the Intel SCU. In addition, the SATA 3.0 ports are connected to the backplane by default. Drives can be configured in RAID 0, 1, 5 and 10 using Intel’s RSTe Software RAID. The 1028U is also equipped with various basic input/output connectivity, including a VGA, COM, and five USB 3.0 ports as well as one IPMI connection to manage the SuperServer when it’s powered off or unresponsive. The 1028U chassis is equipped with a redundant 750W high-efficiency Platinum-level digital power supply, which consists of two hot-plug power modules, so not interruption of the system will occur if one of them needs to be removed.
Our Supermicro SuperServer 1028U-TNR4T+ is configured with 384GB of RAM and Intel E5-2697v3 CPUs with VMware ESXi 6.0 installed. Though our configuration included the base SATA support, it did not, however, include the addon SAS3 card. We also will be populating the SuperServer with two Intel P3700 2TB NVMe SSDs when testing its performance. The Intel SSD Data Center P3700 Series is certainly an impressive line of drives, boasting up to 460,000 in IOPS with Intel also claiming that they can replace the performance of seven SATA SSDs aggregated through an HBA, which translates roughly to 500K IOPS in throughput.
Supermicro SuperServer 1028U-TNR4T+ Specifications
- Form Factor: 1U Rackmount
- Model: CSE-119UAC2-R750
- Motherboard: Super X10DRU-i+
- Drive bays:
- 10x 2.5" Hot-swap drive bays
- Default 8x SATA3 ports and 2x NVMe/SATA3 hybrid ports
- Optional 8x SAS3 ports via AOC
- Intel Xeon processor E5-2600 v3 family (up to 145W TDP)
- Dual Socket R3 (LGA 2011)
- Cores / Cache: Up to 18 Cores / Up to 45MB Cache
- System Bus: QPI up to 9.6 GT/s
- Memory Capacity:
- 24x 288-pin DDR4 DIMM slots
- Up to 1.5TB ECC LRDIMM, RDIMM
- Memory Type: 2133/1866/1600MHz ECC DDR4 SDRAM 72-bit
- DIMM Sizes: 64GB, 32GB, 16GB, 8GB, 4GB
- Memory Voltage: 1.2 V
- Error Detection: Corrects single-bit errors
- Chipset: Intel C612 Express chipset
- SATA: SATA3 (6Gbps) with RAID 0, 1, 5, 10
- Support for Intelligent Platform Management Interface v.2.0
- IPMI 2.0 with virtual media over LAN and KVM-over-LAN support
- ASPEED AST2400 BMC
- Network Connectivity: 4x 10GBase-T ports
- Graphics: ASPEED AST2400 BMC
- Input / Output
- SATA: 10x SATA3 (6Gbps) ports
- SAS: 8x SAS3 ports support via optional Add-on Card
- NVMe: 2x NVMe ports
- 4x RJ45 10GBase-T ports
- 1x RJ45 Dedicated IPMI LAN port
- USB: 5x USB 3.0 ports (2 rear, 2 onboard header, 1 Type A)
- Video: 1x VGA Connector
- Serial Port: 1x Serial header
- BIOS Type: 128Mb SPI Flash EEPROM with AMI BIOS
- BIOS Features:
- Plug and Play (PnP)
- APM 1.2
- PCI 2.3
- ACPI 1.0 / 2.0 / 3.0 / 4.0
- USB Keyboard support
- SMBIOS 2.7.1
- UEFI 2.3.1
- SuperDoctor 5
- Intel® Node Manager 3.0
- Watch Dog
- SMCBU and SMM
- Optional IPMI 1.5 / 2.0
- Power Configurations:
- ACPI Power Management
- Power-on mode for AC power recovery
- Dimensions and Weight
- Height: 1.7" (43mm)
- Width: 17.2" (437mm)
- Depth: 27.82" (707mm)
- Weight: Net Weight: 26 lbs (11.8 kg)
- Gross Weight: 41 lbs (18.6 kg)
- 2x PCI-E 3.0 x16 slots (Full-height, 10.5" L)
- 2x PCI-E 3.0 x8 slots (Low-profile, 1 internal)
- 5x heavy duty fans with optimal fan speed control
- 3x additional fans optional
- 1x Air Shroud
- 750W Redundant AC-DC Power Supplies, +12V/+12Vsb output
- AC Input:
- 100-127Vac, 50-60Hz
- 200-240Vac, 50-60Hz
- Max: 62.5A & Min: 0A (100Vac-127Vac)
- Max: 62.5A & Min: 0A (200Vac-240Vac)
- 12V SB:
- Max: 2.1A
- Min: 0A
Design and build
The Supermicro SuperServer 1028U-TNR4T+ is a 1U form factor server and offers a control panel on the front with power buttons and status monitoring lights in addition to status lights on externally accessible hard drives. The back panel is host to all of the connectivity. As for its design, the SuperServer 1028U has a minimalistic design that makes excellent use of its slim build with 10 hot-swappable 2.5" SAS/SATA drive bays.
The right side of the front panel is home to the Power and UID (unit identifier) buttons, the latter which is used for easy system location in large stack configurations. In addition, there are five LEDs that give users status information concerning their 1028U: Information LED, which alerts users of several states, such as when fans or power failures occur; two NIC LEDs, which indicates network activity on LAN1 and LAN2; and the HDD and Power LEDs, which indicates activity on the drives and the power status, respectively. Also located on the front are two of the four USB 3.0 connectors.
The drive bays are accessed from the front of the chassis without the need to remove the chassis cover or powering off the 1028U. The drives are mounted in drive carriers, which makes their installation and removal fairly painless. In addition, these carriers also improve airflow for the drive bays to help ensure that they remain cool. Supermicro indicates that this is the reason why unpopulated (e.g. no drives) carriers must remain in the chassis during the operation of the 1028U.
The I/O back panel of the 1028U includes a VGA port, two USB 3.0 ports, four 10G Ethernet ports, one dedicated IPMI port, an UID button and LED (both of which are located beside the PCI-E slots). The rear also is home to the 750W high-efficiency Platinum-level digital power supply, which is composed of two separate power modules for power redundancy. This allows users to replace a failed power supply module without having to disrupt or shut down the server. In addition, the PSUs automatically sense the input voltage between 100v to 240v and will operate at that voltage.
Like most Supermicro SuperServers, the 1028U server is easily opened by pressing the two release buttons; slide the cover toward the rear. Once removed, users are greeted with an easily serviceable layout.
Depending on the build, the 1028U is equipped with four to eight 4-cm counter-rotating fans, which provide the cooling for the system. Each fan is made up of two fans joined back-to-back, which rotate in opposite directions. Supermircro designed them this way to generate better airflow as well as to dampen vibration levels.
Application Performance Analysis
In the enterprise market, there is a huge difference between how products claim to perform on paper and how they perform in a live production environment. We understand the importance of evaluating storage as a component of larger systems, most importantly how responsive storage is when interacting with key enterprise applications. To this end, we've rolled out application tests using our MySQL performance via SysBench, which measures average TPS (Transactions Per Second), average latency, as well as average 99th percentile latency over a range of 2 to 32 threads. Here, we will be recording the aggregate performance of the Supermicro 1028U with both a two and four Sysbench virtual machines operating across the NVMe SSDs.
When looking at our average transactions per second benchmark, the Supermicro SuperServer 1028U-TNR4T+ with dual Intel P3700 2TB NVMe SSDs showed a range of 746.92TPS to 5,467.77TPS when testing 2 VMs a the same time. When testing with 4 VMs on one server, the Supermicro 1028U-TNR4T+ showed an increase in throughput to 1,377.32TPS at 2T and 7,789.46TPS at 32T. These results showed an improvement of roughly 84% at lower thread counts, when increasing the amount of virtual machines from 2 to 4, while the higher threadcounts resulted in a 42% improvement with 4 VMs.
With our Sysbench MySQL benchmark running with two VMs, it posted an average latency scaling from 5.36ms up to 11.705ms by the 32nd thread. When increasing the amount of VMs to four, the latency of the Supermicro SuperServer 1028U-TNR4T+ went up slightly, posting 5.81ms at 2 threads, which was an 8% increase, and finishing at 16.43ms under a peak 32T load, which was an increase of 40%.
Looking at our worst-case MySQL latency scenario (99th percentile latency), the SuperServer showed 12.35ms at 2T and scaled up to 23.01ms at 32T when using two VMs on one server. When testing 4 VMs in the same scenario average aggregate 99th percentile latency measured from 13.70ms at 2T up to 32.0ms at 32T. This was an increase of 17% and 39%, respectively.
Equipped with 10 hot-swappable 2.5" drive bays, the Supermicro SuperServer 1028U-TNR4T+ is certainly an impressive enterprise storage server that is offered in a slim, 1U rackmount form factor. The high-end server is comprised of two main subsystems, the SC119UAC(2)-R750 1U chassis and the X10DRU-i+ dual processor serverboard, and offers business with quite a bit of versatility, supporting of both SAS3 12Gb/s and NVMe drives. The NVM Express support makes the 1028U unique due to the drive’s front-serviceability and high-speed, low-latency connections. As a result, this allows the SuperServer to boast vastly improved SSD throughput as well as reduced storage device latency.
We measured the performance of the Supermicro SuperServer 1028U-TNR4T+ by testing it with two 2TB Intel SSD DC P3700 NVMe SSDs. For this review, we used our Percona MySQL database benchmark via SysBench application test, which measures the performance of OLTP activity. The two Intel NVMe drives allowed the 1028U to record an impressive amount of throughput when using both two and four Sysbench VMs, with the latter posting a blistering max aggregate TPS of almost 7,800. This performance was an improvement of roughly 42% compared to the Supermicro configuration with two virtual machines that we tested, which didn't completely max out our CPUs. Moving to even faster processors would allow users to further saturate the I/O potential of the NVMe SSDs.
Though the SuperServer 1028U-TNR4T+ is limited to only two bays for NVMe compatibility, which is something we pointed in our review of the Supermicro SuperServer 1027R-WC1NRT, users are still able to saturate compute resources from just two SSDs. The Supermicro SuperServer certainly takes full advantage of its onboard NVMe flash with reduced latency, increased IOPS, and better power efficiency. In addition, very few companies offer NVMe support to date, so it will be interesting to see how the SuperServer line evolves (and ultimately improves) in the coming years.
- NVMe SSD support offers impressive performance
- Supports up to 1.5TB of RAM allowing for compute-dense configurations
- Attention given to fan design to limit chassis noise and vibration
- Limited to just two NVMe slots
Offered in a slim, 1U rackmount form factor, the Supermicro SuperServer 1028U-TNR4T+ is an impressive enterprise storage server that offers enterprise buyers a ton of flexibility and performance with its NVMe compatibility.