The Lenovo ThinkSystem SR665 is a feature-rich, 2nd generation AMD EPYC dual-socket 2U server, supporting end-to-end PCIe Gen4 support. Similar to its smaller 1U sibling the SR635, it also supports a wide range of storage configuration options, including Gen4 U.2 NVMe SSDs. Being an AMD EPYC server comes with several advantages including typically higher performance, faster RAM, more DIMMs, and PCIe Gen4 support. This server is ideal for Inference, virtualization, VDI, HPC, and Hyperconverged infrastructure.
The Lenovo ThinkSystem SR665 is a feature-rich, 2nd generation AMD EPYC dual-socket 2U server, supporting end-to-end PCIe Gen4 support. Similar to its smaller 1U sibling the SR635, it also supports a wide range of storage configuration options, including Gen4 U.2 NVMe SSDs. Being an AMD EPYC server comes with several advantages including typically higher performance, faster RAM, more DIMMs, and PCIe Gen4 support. This server is ideal for Inference, virtualization, VDI, HPC, and Hyperconverged infrastructure.
AMD EPYC 7002 brings several benefits over its competitors. Right off the bat, they tend to have more cores and threads and bring more performance. They support faster and more DRAM and support PCIe Gen4 technology (end-to-end in this case) that in turn can bring massive performance increases. The high core count can also help with cost savings for certain licensing models, driving the performance of a dual-socket system with a single CPU or a quad-socket platform with two CPUs. The SR665 takes full advantage of everything above.
Specifically, the Lenovo ThinkSystem SR665 supports CPUs up to 64 cores and 128 threads with core speed up to 3.7GHz. For memory, the SR665 supports up to 32 TruDDR4 memory DIMMs, with 8 memory channels and 2 DIMMs per channel. With 1 DIMM installed per channel (8 DIMMs total), memory operates at up to 3200 MHz. With 2 DIMMs per channel (32 DIMMs total), memory operating at up to 2933 MHz. The server can support up to 4TB of DRAM. For GPUs, the SR665 can support up to 3 double-wide or 8 single-wide GPUs.
Storage, and specifically storage configurability, is really interesting on the Lenovo ThinkSystem SR665 Server. According to Lenovo, the server offers 28 different drive bay configurations in the front, middle, and rear of the server and 5 different slot configurations at the rear of the server.
For 2.5-inch drives, the server can support up to 40 through a combination of front, mid, and rear. The server supports up to 16 NMVe SSDs without oversubscription of PCIe lanes and up to 32 with 1:2 oversubscription. Conversely, the SR665 can support up to 20 3.5-inch drives for maximum storage capacity. For M.2, the server supports either one drive or two in RAID1 for boot or data storage.
For management, the SR665 uses a combination of Lenovo XClarity management, ThinkShield security features, and Lenovo Services. The idea is to provide powerful, simple, and secure management. We did a fairly deep dive on XClarity linked above, for those looking to learn a bit more.
Lenovo ThinkSystem SR665 Specifications
| Form Factor | 2U rack server |
| Processors | Up to two (2) AMD EPYC 7002 Generation Processors, up to 64C, 280W |
| Memory | 32x DDR4 memory slots; Maximum 4TB using 128GB RDIMMs; up to 1DPC at 3200MHz, 2DPC at 3200MHz |
| Drive Bays | Up to 20x 3.5-inch or 40x 2.5-inch drives; Maximum of 32x NVMe drives with 1:2 connection |
| Expansion Slots | Up to 8x PCIe 4.0 slots, 1x OCP 3.0 adapter slot |
| GPU | Up to 8x single-width GPUs or 3x double-width GPUs |
| Network Interface | OCP 3.0 mezz adapter, PCIe adapters |
| Power | Dual redundant PSUs (up to 1800W Platinum) |
| Ports | Front: 1x USB 3.0, 1x USB 2.0, 1x VGA (optional) Rear: 3x USB 3.1, 1x Serial port (optional), 1x RJ-45 (management) |
| Systems Management | Lenovo XClarity Controller |
| OS Support | Microsoft Windows Server, SUSE Linux Enterprise Server, Red Hat Enterprise Linux, VMware ESXi |
| Limited Warranty | 1- and 3-year customer-replaceable unit and onsite services, next business day 9×5, optional service upgrades |
Lenovo ThinkSystem SR665 Design and Build
The Lenovo ThinkSystem SR665 is a 2U server that has the same overall look as all the ThinkSystem portfolio or Lenovo in general, primarily black with red highlights. Across the front of the server as the 24 drive bays. The left side has an external diagnostics port and an optional VGA port. The right side has two USB 3.1 ports, an XXC port, a power button, and indicator lights.
The rear of the device supports up to 8 PCIe slots across the top. Across the bottom is the OCP 3.0 slot, an error LED, a management Ethernet port, Locator LED, an XCC management port, a video port, three more Gen1 USB 3.1 ports, an NMI, and two hot-swappable PSUs.
Under the hood, we can see the airflow cover over the two CPUs and RAM DIMMs. In front of them are the fans and behind them are the risers for the PCIe slots. The M.2 boot card and RAID card power backup attach to the top of the cover.
The M.2 boot module mounted on the air baffle offers fast flash storage for system storage, without consuming a primary storage bay for low-priority tasks.
Lenovo ThinkSystem SR665 Performance
Lenovo ThinkSystem SR665 configuration:
- 2 x AMD EPYC 7742
- 512GB, 256GB per CPU
- VDbench Performance Storage: 8 x 3.84Tb Micron 9300 (Gen3) or 8 x 3.84TB SK hynix PE8010 (Gen4)
- SQL Server and Sysbench Storage: 8 x 3.84Tb Micron 9300 (Gen3)
- CentOS 8 (2004)
- ESXi 6.7u3
SQL Server Performance
StorageReview’s Microsoft SQL Server OLTP testing protocol employs the current draft of the Transaction Processing Performance Council’s Benchmark C (TPC-C), an online transaction processing benchmark that simulates the activities found in complex application environments. The TPC-C benchmark comes closer than synthetic performance benchmarks to gauging the performance strengths and bottlenecks of storage infrastructure in database environments.
Each SQL Server VM is configured with two vDisks: 100GB volume for boot and a 500GB volume for the database and log files. From a system resource perspective, we configured each VM with 16 vCPUs, 64GB of DRAM and leveraged the LSI Logic SAS SCSI controller. While our Sysbench workloads tested previously saturated the platform in both storage I/O and capacity, the SQL test looks for latency performance.
This test uses SQL Server 2014 running on Windows Server 2012 R2 guest VMs, and is stressed by Dell’s Benchmark Factory for Databases. While our traditional usage of this benchmark has been to test large 3,000-scale databases on local or shared storage, in this iteration we focus on spreading out four 1,500-scale databases evenly across our servers.
SQL Server Testing Configuration (per VM)
- Windows Server 2012 R2
- Storage Footprint: 600GB allocated, 500GB used
- SQL Server 2014
-
- Database Size: 1,500 scale
- Virtual Client Load: 15,000
- RAM Buffer: 48GB
- Test Length: 3 hours
- 2.5 hours preconditioning
- 30 minutes sample period
For SQL Server average latency the SR665 had no problem maxing out this workload, with an aggregate of 1ms average and individual VMs of 1ms each.
Sysbench MySQL Performance
Our first local-storage application benchmark consists of a Percona MySQL OLTP database measured via SysBench. This test measures average TPS (Transactions Per Second), average latency, and average 99th percentile latency as well.
Each Sysbench VM is configured with three vDisks: one for boot (~92GB), one with the pre-built database (~447GB), and the third for the database under test (270GB). From a system resource perspective, we configured each VM with 16 vCPUs, 60GB of DRAM and leveraged the LSI Logic SAS SCSI controller.
Sysbench Testing Configuration (per VM)
- CentOS 6.3 64-bit
- Percona XtraDB 5.5.30-rel30.1
-
- Database Tables: 100
- Database Size: 10,000,000
- Database Threads: 32
- RAM Buffer: 24GB
- Test Length: 3 hours
- 2 hours preconditioning 32 threads
- 1 hour 32 threads
With the Sysbench OLTP we saw an aggregate score of 17,238 TPS for 8VMs and 32,649 TPS for 16VMs.
With Sysbench average latency we saw aggregate scores of 14.85ms for 8VMs and 15.68ms for 16VMs.
For our worst-case scenario latency (99th percentile) the SR665 had aggregate scores of 30.27ms for 8VMs and 30.12ms for 16VMs.
VDBench Workload Analysis
When it comes to benchmarking storage arrays, application testing is best, and synthetic testing comes in second place. While not a perfect representation of actual workloads, synthetic tests do help to baseline storage devices with a repeatability factor that makes it easy to do apples-to-apples comparison between competing solutions.
These workloads offer a range of different testing profiles ranging from “four corners” tests, common database transfer size tests, as well as trace captures from different VDI environments. All of these tests leverage the common vdBench workload generator, with a scripting engine to automate and capture results over a large compute testing cluster. This allows us to repeat the same workloads across a wide range of storage devices, including flash arrays and individual storage devices.
Profiles:
- 4K Random Read: 100% Read, 128 threads, 0-120% iorate
- 4K Random Write: 100% Write, 64 threads, 0-120% iorate
- 64K Sequential Read: 100% Read, 16 threads, 0-120% iorate
- 64K Sequential Write: 100% Write, 8 threads, 0-120% iorate
- Synthetic Database: SQL and Oracle
- VDI Full Clone and Linked Clone Traces
Here we did a comparison of PCIe Gen3 and Gen4 drives, Micron 9300 vs SK hynix PE8010. Not so much to see which of the drives are better, but instead to see how each type of technology will perform in the server. Visible below are our eight Micron 9300 NVMe SSDs sitting in the SR665 AnyBay slots, with a hard drive above for reference.
With random 4K read, the Lenovo ThinkSystem SR665 with the SK Hynix Gen4 drive started at 75.9µs latency and 412,428 IOPS before going on to peak at 3,851,738 IOPS and a latency of 170.2µs. The Micron 9300 Gen3 drive peaked higher with 4,076,949 IOPS but a higher latency of 197.2µs.
4K random write saw the Gen4 drive perform dramatically better in the SR665 staying under 100µs until roughly 2.6 million IOPS and went on to peak at 2,898,564 IOPS with a latency of 154.4µs. The Gen3 drives saw a peak of about 1.2million IOPS at roughly 200µs before dropping off some.
Switching over to 64K sequential work, the Gen4 drives peaked at 521,788 IOPS or 32.6GB/s at a latency of 366.1µs. The Gen3 drives peaked at 425,209 IOPS or 26.6GB/s at a latency of 545µs.
64K write for the SR665, the Gen4 drives peaked at 213,099 IOPS or 13.3GB/s at a latency of 442.1µs before dropping off some. The Gen3 drives peaked at about 123K IOPS or 7GB/s at a latency of roughly 460µs before dropping off some.
Our next set of tests are our SQL workloads: SQL, SQL 90-10, and SQL 80-20. Starting with SQL, the SK hynix Gen4 drive stayed below 100µs until about 1.75 million IOPS and went on to peak at 2,111,582 IOPS with a latency of only 113.6µs. The Gen3 peaked at 1,602,112 IOPS with a latency of 153.4µs.
For SQL 90-10 the Gen4 drives again started and stayed under 100µs, this time until they broke about 1.2 million IOPS. The drives went on to peak at 1,769,165 IOPS with a latency of 130.7µs before a slight drop off. Gen3 SSDs also fared well with a peak of 1,441,311 IOPS and a latency of 156.8µs.
SQL 80-20 saw the Gen4 drives in the SR665 peak at 1,657,107 IOPS with a latency of 141.9µs. The Gen3 drives were able to hit a peak of 1,246,384 IOPS with a latency of 168.9µs.
Next up are our Oracle workloads: Oracle, Oracle 90-10, and Oracle 80-20. Starting with Oracle, the Lenovo ThinkSystem SR665 with Gen4 SSDs start under 100µs and stayed there until they broke about 900K IOPS. The drives went on to peak at 1,484,001 IOPS with a latency of 154.4µs. For the Gen3 drives, the server gave us a peak performance of 1,140,743 IOPS and a latency of 178.8µs.
For Oracle 90-10 the Gen4 drives had a peak of 1,511,538 IOPS at a latency of 110.5µs. The Gen3 drives had a strong showing as well with a peak of 1,231,073 IOPS and a latency of 134.8µs.
Oracle 80-20 saw the peak performance of the Gen4 drives at 1,369,215 IOPS and a latency of 117.7µs. The Gen3 SSDs hit 1,089,788 IOPS and a latency of 141.6µs.
Next, we switched over to our VDI clone test, Full and Linked. For VDI Full Clone (FC) Boot, the SK hynix drives on the SR665 started off with sub-100µs latency performance until 700K IOPS and peaked at 1,379,893 IOPS and a latency of 174.6µs. The Micron Gen3 drives peaked at 1,185,395 IOPS and a latency of 198.1µs before dropping off some.
With VDI FC Initial Login we see quite a bit of difference between the two generations of PCIe SSDs. The Gen4 began with latency under 100µs until about 400K IOPS and peaked at 636,576 IOPS with a latency of 268.3µs. The Gen3 peaked at 324,073 IOPS with a latency of 356µs.
VDI FC Monday Login again saw a larger difference with the Gen4 drives peaking at 526,115 IOPS at a latency of 179.5µs and the Gen3 peaking at 293,012 IOPS at 235.5µs for latency.
Switching to VDI Linked Clone (LC) Boot, the SR665 loaded with the Gen4 drives peaked at 684,910 IOPS at a latency of 149.6µs. The Gen3 drives hit of peak of 538,330 IOPS at a latency of 189µs.
The VDI LC Initial Login again saw a bigger difference between the two generations with the Gen4 peaking at 288,211 with a latency of 156µs and the Gen3 peaking at 161,852 IOPS and a latency of 227.8µs.
Finally, with VDI LC Monday Login we saw the Gen4 drives go to a peak of 386,865 IOPS and a latency of 237µs and the Gen3 drives hit 181,515 IOPS with a latency of 309.3µs.
Conclusion
The Lenovo ThinkSystem SR665 is a 2U server that can house and take advantage of two AMD EPYC Gen2 CPUs. Being an AMD EPYC-based server lends several benefits such as the ability to use PCIe Gen4 technology in storage, GPUs, and FPGAs as well as faster RAM. From a hardware perspective, the server can house up to two AMD EPYC CPUs supporting up to 64 cores and 128 threads with core speed up to 3.7GHz, up to 4TB of RAM (speeds up to 3200 MHz), and has 9 slots for expansion.
The SR665 has lots of storage configurability as well. The server has front, mid, and rear drive bay capabilities and can support up to 40 2.5-inch drives. The server supports NVMe and SATA drives. Leveraging XClarity for management, the SR665 is aimed at Inference, virtualization, VDI, HPC, and Hyperconverged infrastructure use cases.
To test performance, we ran both our Application Workload Analysis as well as our VDBench. In our Application Workload Analysis, we ran SQL Server average latency and our Sysbench tests. In our SQL Server average latency, the SR665 had an aggregate latency of 1ms. In Sysbench we ran 8VM and 16VM. In OLTP we saw aggregate scores of 17,238 TPS for 8VMs and 32,649 for 16VMs, 14.85ms for 8VMs and 15.68ms for 16VMs for average latency, and 30.27ms for 8VMs and 30.12ms for 16VMs for worst-case scenario latency.
The Lenovo ThinkSystem SR665 can leverage both PCIe Gen3 and Gen4 devices. This means that users looking to deploy a SR665 in their environments may use one or the other. In order to give a better overall view of how the server works, we put both types of storage in and tested it. To be clear, this isn’t a “which one is better” type of test. While the Gen4 storage isn’t quite saturated with offerings just yet, it is typically faster than Gen3 in nearly every case. So Gen4 will be faster in most instances. However, with both drives tested, users will know what to expect with what type of storage they choose to leverage, e.g., if they are using Gen3 now they will know what to expect and if they change to Gen4 in the future they will know what the results should be.
Before diving into the performance section, it’s important to note that we just scratched the surface of what’s possible with the SR665. We have a 16 bay configuration, supporting only 8 NVMe devices. Lenovo offers many configuration options, including some that go up to 16 NVMe bays. So for even higher potential, you can further spec out this server beyond what we were able to look at.
In our 4K read we saw peaks of the Gen4 hit 3.9 million IOPS and the Gen3 hit 4.1 million IOPS, with Gen4 leading with a 14% reduction in latency, for 4K write the peaks were 2.9 million IOPS in Gen4 and 1.2 million IOPS in Gen3, for sequential work we saw highs of 32.6GB/s for Gen4 and 26.6GB/s write in 64K read, and in 64K write the peaks were 13.3GB/s for Gen4 and 7GB/s for Gen3. Our next test is SQL, SQL 90-10 and SQL 80-20 where the server gave us 2.1 million IOPS for Gen4 and 1.6 million IOPS for Gen3, 1.8 million IOPS for Gen4 and 1.4 million IOPS for Gen3 with SQL 90-10, and SQL 80-20 hit 1.7 million IOPS for Gen4 and 1.2 million for Gen3.
The next batch of tests were Oracle, Oracle 90-10, and Oracle 80-20 where the S665 hit 1.5 million IOPS for Gen4 and 1.1 million IOPS for Gen3, Oracle 90-10 saw 1.5 million IOPS for Gen4 and 1.2 million IOPS for Gen3, and 1.4 million IOPS for Gen4 and 1.1 million for Gen3 for Oracle 80-20. The last test we saw the million IOPS barrier broken was the VDI FC Boot with both the Gen4 (1.4 million IOPS) and the Gen3 (1.2 million IOPS).
The Lenovo ThinkSystem SR665 is a highly configurable 2U server that can really bring out the performance capabilities with the AMD EPYC CPUs. Whether users are packing it with PCIe Gen3 or Gen4, they will be impressed with its capabilities.
Engage with StorageReview
Newsletter | YouTube | Podcast iTunes/Spotify | Instagram | Twitter | Facebook | RSS Feed
