September 17th, 2018 by Marshall Gunnell
Mellanox ConnectX-5 EN Dual-Port 100GbE DA/SFP NIC Review
The Mellanox ConnectX-5 EN is a dual port network interface card (NIC) designed to deliver extreme bandwidth at sub-600 nanosecond latency and a high message rate with its 100GbE transfer rate. With its advanced storage capabilities including NVMe-oF target offloads, this NIC is ideal for High Performance, Cloud, Data Analytics and Storage platforms.
The latest version of ConnectX controller, version -5, delivers high bandwidth, lower latency, and high computation efficiency making it an ideal choice in HPC infrastructures. The NIC leverages RDMA over Converged Ethernet (RoCE) technology in order to hit the low latency and high performance. The ConnectX-5 also provides MPI and SHMEM/PGAS and Rendezvous Tag Matching offload as enhancements to HPC infrastructures. The new controller supports Switch Adaptive-Routing and data delivered out-of-order, enhancing RDMA network capabilities.
While the new controller in this NIC comes with the above, where it gets interesting for storage is its support for NVMe over Fabric (NVMe-oF). NVMe is becoming the go to technology when fast storage is needed. While it has promise in several areas, NVMe-oF is particularly interesting to us for its promise of performance and low latency. The ConnectX-5 provides NVMe-oF target offloads, meaning NVMe storage access without CPU intervention for even better performance and lower latency.
Mellanox ConnectX-5 EN specifications
- Device Type: Network Adaptor
- Form Factor: Plug-In Card
- Interface: PCIe
- Ports: 100 Gigabit Ethernet x 2
- Data Link Protocol: 100 Gigabit LAN
- Data Transfer Rate: 100Gbps
- Expansion / Connectivity
- Interfaces: 2 x 100Gb Ethernet
Design and Build
The Mellanox ConnectX-5 EN is low profile, dual Ethernet configuration, plug in network interface card. With its screwless design, the card is easily installed inside a compatible server. At the front of the card, you’ll find two QSFP+ ports and, at the back, a large black heat sink covering the majority of the Mellanox controller.
The back of the card has a couple of identification stickers, leaving most of this side of the card blank, as you’d come to expect.
The Mellanox ConnectX-5 EN Dual-Port 100GbE DA/SFP is a PCIe NIC ideal for performance-demanding environments. Its two 100GbE ports are also backward compatible with 50GbE/40Gbe/25Gbe/ and 10GbE, allowing for flexible network upgrade capabilities as the needs arise. Offering high bandwidth, sub-600 nanosecond latency, and high message rate, the ConnectX-5 EN is ideal for building a bottleneck-free storage system for achieving extreme efficiency on a variety of workloads in a virtualized environment.
The Mellanox ConnectX-5 EN, which we integrated into our Dell R740xds, has been an invaluable asset to the StorageReview lab, playing a key role in the testing of NVMeoF products, such as our review of Toshiba's KumoScale. During our KumoScale test that leveraged the ConnectX-5 NIC we saw performances of nearly 3 million IOPS for 4K read and 64k read speeds of 13.36GB/s. The software and NIC were able to hit over a million IOPS on several of our tests.
Anyone that is looking to leverage NVMe-oF or utilize HPC should take a long hard look at a NIC such as the Mellanox ConnectX-5 EN Dual-Port 100GbE card.
Discuss This Review