The ASRock Rack 4U36L6E-MILAN2/2T is a 4U rack server with support for two AMD EPYC CPUs, 36 3.5″ bays (4 of which can be used with NVMe SSDs), two 2.5″ NVMe boot drive bays and a pair of M.2 slots on the board for good measure. With this much compute and storage potential, this server can take on a variety of personas, from a traditional high-density storage server to a mixed-use virtualization box.
The ASRock Rack 4U36L6E-MILAN2/2T is a 4U rack server with support for two AMD EPYC CPUs, 36 3.5″ Bays (4 of which can be used with NVMe SSDs), two 2.5″ NVMe boot drive bays and a pair of M.2 slots on the board for good measure. With this much compute and storage potential, this server can take on a variety of personas, from a traditional high-density storage server to a mixed-use virtualization box.
We’ve been working with ASRock Rack quite a bit lately and recently did a deep dive into their new midrange SMB server. That 2U server sported an Intel Xeon Scalable Processor and offered a highly flexible build in a 12-bay chassis with optional NVMe bays. Where the 2U server addressed a variety of typical use cases, the 4U AMD server, reviewed here, elevates the power and density to another level.
With so much storage and the ability to accelerate workloads with NVMe SSDs, the 4U36L6E-MILAN2/2T will fit a variety of likely use cases. Service providers will opt to fill all of these bays from the outset, using some of the flash to accelerate HDD-based workloads. On the other hand, SMBs with a growing data footprint can start partially filled, use the NVMe bays for high-performance applications needs, and expand storage as business demands change. Either way, ASRock Rack customers will appreciate the flexibility available on this server.
In our case, we focused on a virtualized environment with VMware ESXi. Users can just as easily leverage Hyper-V, Proxmox, or other hypervisors that fit in their environments. Or, forgo the hypervisor entirely and leverage a software-defined solution like TrueNAS, OpenStack, OpenShift, or Cloudian to solve more specific needs. The advantage of any of these options is massive when you factor in the compute potential of the AMD EPYC Gen3 CPUs, offering a combined 128 cores in some configurations.
ASRock Rack 4U36L6E-MILAN2/2T Hardware Overview
With its dual-socket SP3 (LGA 4094) design, the 4U36L6E-MILAN2/2T server supports AMD EPYC 7003 and 7002 series processors and 8+8 DIMM slots, providing the necessary processing power to handle a wide range of workloads.
While the processing power is an essential component, the other major highlight of the 4U36L6E-MILAN2/2T is its storage capacity. The server features 32 hot-swap 3.5″ SATA/SAS drive bays on the front side, as well as four hot-swap 3.5″ SATA/SAS drive bays or 4 hot-swap 2.5″ NVMe (PCIe3.0 x4) drive bays. From the back, there are an additional 12 hot-swap 3.5″ SATA/SAS drive bays and two hot-swap 2.5″ NVMe drive bays, and inside there are an additional set of two M.2 SSD slots.
Getting that data in and out requires good connectivity options. The server comes equipped with dual 10GbE Intel X550-AT2 NICs on board. Of course, additional NICs can be added to the PCIe slots if needed.
In terms of expandability, the 4U36L6E-MILAN2/2T offers several options thanks to six low-profile PCIe 4.0 expansion slots, allowing businesses to add additional hardware as needed. While onboard 10GbE is included, these slots can be leveraged for 25/50/100/200GbE interfaces, small GPUs like the popular NVIDIA A2, or other expansion devices, depending on the exact deployment configuration needed. Of the available six slots, one is needed for a RAID card or HBA for the SATA/SAS connectivity of the server.
For remote management, ASRock Rack includes an HTML5-based WebGUI for monitoring the server and performing maintenance functions. The layout is intuitive, especially for day 0 activities where you are getting an operating system deployed on the server.
ASRock Rack 4U36L6E-MILAN2/2T Specifications
| System | |
| Form Factor | 4U Rackmount |
| Dimensions | 699 x 432 x 176 mm (27.5″ x 17″ x 6.9″) |
| Motherboard | ROME2D16-2T |
| Front Panel | |
| Buttons | System reset, Power |
| LEDs | Power, LAN1 activity, LAN2 activity, System failure, HDD status |
| I/O Ports | 2 Type-A (USB3.2 Gen1) |
| External Drive Bay / Storage | |
| Front Side Bays | 20 Hot-swap 3.5″ SATA/SAS drive bays* 4 Hot-swap 3.5″ SATA/SAS drive bays or 4 Hot-swap 2.5″ NVMe (PCIe3.0 x4) drive bays *Additional RAID/HBA card required to support the 36 x 3.5″ SATA/SAS drives |
| Front Backplane | 24-port 12Gbps active backplane, 4-port PCIe3.0 backplane |
| Rear Side Bays | 12 Hot-swap 3.5” SATA/SAS drive bays, 2 Hot-swap 2.5” NVMe (PCIe3.0 x4) drive bays |
| Rear Backplane | 12-port 12Gbps active backplane, 2-port PCIe3.0 backplane |
| Internal Storage | 1 M-key (PCIe4.0 x4 or SATA 6Gb/s), supports 22110/2280/2242 form factor [CPU0] 1 M-key (PCIe4.0 x4), supports 22110/2280/2242 form factor [CPU0] |
| Power Supply | |
| Type | 1+1 Redundant CRPS |
| Output Watts | 800W @ 100-127Vac input / 1200W @ 200-240Vac input |
| Efficiency | 80-PLUS Platinum |
| System Fan | |
| Fan | 7 Hot-swap 80x38mm fans |
| Processor System | |
| CPU | Supports AMD EPYC 7003 /7002 Series Processors |
| Socket | Dual Socket SP3 (LGA 4094) |
| Thermal Design Power | Up to 205W |
| Chipset | System on Chip |
| System Memory | |
| Supported DIMM Quantity | 8+8 DDR4 288-pin DIMM slots (1DPC) |
| Supported Type | DDR4 288-pin RDIMM, LRDIMM |
| Max. Capacity per DIMM | RDIMM: up to 64GB, LRDIMM: up to 256GB |
| Max. DIMM Frequency | RDIMM/ LRDIMM: up to 3200 MHz |
| Voltage | 1.2V |
| PCIe Expansion Slots (SLOT7 close to CPU) | |
| PCIe x 16 | Slot 6: low-profile PCIe 4.0 x16 [CPU1] Slot 5: low-profile PCIe 4.0 x16 [CPU0] Slot 4: low-profile PCIe 4.0 x16 [CPU1] Slot 3: low-profile PCIe 4.0 x16 [CPU0] Slot 1: low-profile PCIe 4.0 x16 [CPU0] |
| PCIe x 8 | Slot 2: low-profile PCIe 4.0 x8 [CPU1] |
| Server Management | |
| BMC Controller | ASPEED AST2500: IPMI2.0 with iKVM and vMedia support |
| Rear I/O | |
| UID Button/LED | 1 UID w/ LED |
| VGA Port | 1 DB15 (VGA) |
| USB 3.2 Gen1 Port | 2 Type-A (USB3.2 Gen1) |
| RJ45 | 2 RJ45 (10GbE) by Intel X550-AT2 1 x Realtek RTL8211E for dedicated management GLAN |
| System BIOS | |
| BIOS Type | 32MB AMI UEFI BIOS |
| BIOS Features | Plug and Play (PnP), ACPI 6.3 Compliance Wake-Up Events, SMBIOS 3.3.0, ASRock Rack Instant Flash |
| Hardware Monitor | |
| Temperature | CPU, MB, Card side, X550 Temperature Sensing |
| Fan | Fan Tachometer CPU Quiet Fan (Allow Chassis Fan Speed Auto-Adjust by CPU Temperature) Fan Multi-Speed Control |
| Voltage | Vsoc,Vcpu, VCCM, +3V,+5V, +12V, BAT, +1.8VSB, +3VSB, +5VSB |
| Environment | |
| Temperature | Operation temperature: 10°C ~ 35°C / Non-operation temperature: -40°C ~ 70°C |
ASRock Rack 4U36L6E-MILAN2/2T Virtualization Testing
We configured the ASRock Rack 4U36L6E-MILAN2/2T as a VMware ESXi host to illustrate running capacity- and I/O-dense workloads. We installed 20 22TB HDDs split into two pools and four 7.68TB Solidigm P5520 NVMe SSDs to run workloads individually.
We then migrated four SQL Server VMs, eight MySQL Sysbench VMs, and 16 storage loadgen VMs onto the server to measure system performance. These workloads show the I/O and CPU potential of the platform, as well as storage capacity benefits for the hard drive RAID groups.
ASRock Rack 4U36L6E-MILAN2/2T Testing Configuration
- 2 x AMD EPYC 7763 64-Core CPUs
- 256GB DDR4
- 8 x 22TB WD Gold
- 12 x 22TB WD Red Pro
- 4 x 7.68TB Solidigm P5520
- Broadcom 9460-16i
- VMware ESXi 7.0.3
SQL Server Performance
Each SQL Server VM is configured with two vDisks: a 100GB volume for boot and a 500GB volume for database and log files. From a system resource perspective, we configured each VM with eight vCPUs, and 64GB of DRAM and leveraged the LSI Logic SAS SCSI controller. While our Sysbench workloads tested previously saturated the platform in storage I/O and capacity, the SQL test looks for latency performance.
This test uses SQL Server 2014 running on Windows Server 2012 R2 guest VMs and is stressed by Quest’s Benchmark Factory for Databases. StorageReview’s Microsoft SQL Server OLTP testing protocol employs the current draft of the Transaction Processing Performance Council’s Benchmark C (TPC-C). This online transaction-processing benchmark simulates the activities found in complex application environments. The TPC-C benchmark comes closer than synthetic performance benchmarks to gauging the performance strengths and bottlenecks of storage infrastructure in database environments. Each instance of our SQL Server VM for this review uses a 333GB (1,500 scale) SQL Server database and measures the transactional performance and latency under a load of 15,000 virtual users.
SQL Server Testing Configuration (per VM)
- Windows Server 2012 R2
- Storage Footprint: 600GB allocated, 500GB used
- SQL Server 2014
- Database Size: 1,500 scale
- Virtual Client Load: 15,000
- RAM Buffer: 48GB
- Test Length: 3 hours
- 2.5 hours preconditioning
- 30 minutes sample period
We positioned four SQL Server VMs onto the ASRock Rack 4U36L6E-MILAN2/2T, with each VM on a dedicated SSD. In this test, the server had no problem keeping up with the demand and maxed out the test with a latency of 1ms for each VM.
Sysbench Performance
The next application benchmark consists of a Percona MySQL OLTP database measured via SysBench. This test also measures average TPS (Transactions Per Second), average latency, and average 99th percentile latency.
Each Sysbench VM is configured with three vDisks: one for boot (~92GB), one with the pre-built database (~447GB), and the third for the database under test (270GB). From a system resource perspective, we configured each VM with eight vCPUs, and 60GB of DRAM and leveraged the LSI Logic SAS SCSI controller.
Sysbench Testing Configuration (per VM)
- CentOS 6.3 64-bit
- Percona XtraDB 5.5.30-rel30.1
- Database Tables: 100
- Database Size: 10,000,000
- Database Threads: 32
- RAM Buffer: 24GB
- Test Length: 3 hours
- 2 hours preconditioning 32 threads
- 1 hour 32 threads
For our Sysbench test, we used eight VMs, with two VMs positioned on each SSD VMware datastore. With this eight VM load, we could scale up to 23,800TPS, averaging 2,975TPS per VM.
We measured average latency across the Sysbench VM group between 10.65ms and 10.85ms.
Measuring 99th percentile latency in the Sysbench 8VM group, the aggregate average measured 20.45ms.
VDBench Workload Analysis
We use VDbench inside a VMware ESXi environment to show the performance attained from storage across multiple VMs sharing common datastores. The test leveraged a group of 16 storage loadgen VMs built for testing shared storage in a virtualized environment. Each VM is then provisioned with two 125GB vmdks, totaling a 4TB storage footprint. As we look at two types of storage in this server, we focus on the higher capacity storage from our two HDD pools and the performant storage provided by four 7.68TB NVMe SSDs. We used two RAID6 pools for our HDD tests with read-ahead and write-back enabled.
Profiles:
- 2048K Sequential: 100% Read, 100% Write, four threads, 0-120% iorate
- 64K Sequential: 100% Read, 100% Write 32 threads, 0-120% iorate
- 4K Random: 100% Read, 100% Write 128 threads, 0-120% iorate
From our 16 Vdbench guest VMs running inside our ESXi environment, we first look at the 2048k sequential transfer performance seen in aggregate. Here we measure a peak of 2,176MB/s in our write workload.
Switching to our read workload with the 2048k transfer size, we measure an aggregate speed of 2,064MB/s.
Next, we turn to the NVMe SSDs with a smaller 64K sequential transfer workload. In our first test measuring write speeds, we measure a peak of 13.4GB/s.
Switching to read bandwidth, we measured a peak of 14GB/s.
Our last test segment focuses on random 4K transfer speeds, first looking at write performance. Across our 16VMs, we measure an aggregate 4K write speed of 694k IOPS.
Switching to 4K random read, we measure a peak of 745k IOPS.
Overall this is just one scenario showing the performance potential of the ASRock Rack 4U36L6E-MILAN2/2T as a VMware ESXi host. However, this platform can run many different operating systems and configurations.
Final Thoughts
The ASRock Rack 4U36L6E-MILAN2/2T is a powerful and versatile high-density storage server well-suited for various applications, including virtualization, software-defined storage, and traditional high-density storage. It features dual AMD EPYC CPUs and support for up to 36 3.5″ drives, including four NVMe SSDs, two 2.5″ NVMe boot drive bays, and a pair of M.2 slots on the motherboard.
With its high storage capacity and the ability to accelerate workloads with NVMe SSDs, the 4U36L6E-MILAN2/2T is ideal for service providers and small and medium-sized businesses with growing data storage needs. It also offers six low-profile PCIe 4.0 expansion slots, dual 10GbE Intel X550-AT2 NICs, and an HTML5-based WebGUI for remote management.
From the performance angle, the ASRock Rack 4U36L6E-MILAN2/2T has several tricks up its sleeve. It blends high-density platter storage with dual-socket AMD EPYC Gen3 processors and tops it off with up to six Gen4 NVMe U.2 bays. In our virtualized tests, we found the platform had no trouble keeping up with tasking database workloads, offering just under 24k TPS in Sysbench without breaking a sweat and maxing out our SQL Server workload at 1ms across each instance. This was done leveraging just four NVMe bays while leaving 32 3.5″ bays available for high-capacity hard drives.
For analytical work with large datasets, this platform has the potential to run tasking workloads internally and reach into large datasets without needing to draw from an external shared storage platform. If you wanted to push this box to the max, you could stuff 704TB of spinning media into the server and still leave room for plenty of NVMe flash.
In total, the ASRock Rack 4U36L6E-MILAN2/2T is an excellent choice for those looking to maximize storage capacity with a nod toward performance as well. We’re quite pleased with the use of NVMe in this system; having access to six U.2 SSDs expands the options for accelerated HDD workloads in a software-defined system.
View the ASRock Rack 4U36L6E-MILAN2/2T information on their website. Contact the ASRock Rack Sales team for pricing and availability at [email protected].
This report is sponsored by ASRock Rack. All views and opinions expressed in this report are based on our unbiased view of the product(s) under consideration.
Engage with StorageReview
Newsletter | YouTube | Podcast iTunes/Spotify | Instagram | Twitter | TikTok | RSS Feed
