October 28th, 2013 by StorageReview Enterprise Lab
Dell EqualLogic PS6110XS Review
The Dell EqualLogic PS6110XS is a 24-bay SAN that derives its storage capacity and performance from 17 x 2.5" SFF 600GB 10K RPM SAS HDDs and 7 x 2.5" 400GB SSDs and features dual storage controllers, each with 4GB of memory. As with other offerings in the Dell EqualLogic family, the PS6110XS reflects a major commitment from Dell to both advanced functionality and ease of use. The PS6110XS's ease of use and management is highlighted by a few key features like automated load balancing, to allow data migration across free storage, as well as intelligent data pooling that tiers data between the SSDs and HDDs based on usage.
The Dell PS6110 family is available in several configurations. There are PS6110 systems with 2.5-inch HDDs as well as with LFF 3.5-inch HDDs. Our own PS6110XS unit allows hybrid array configurations that combine 17 hot-pluggable 2.5-inch 10K SAS HDDs and 7 hot-pluggable SSDs. Each controller on the PS6110XS features a 10GBase-T with RJ-45 connectivity as well as a 10GbE SFP+ port for fiber and Twinax. Each control module incorporates its own Ethernet and serial ports for management, as well as implements Dell’s “cache to flash” functionality for data protection.
Earlier this year, we had an opportunity to visit Dell’s EqualLogic lab to get our first look at the PS6110XS in action, as well as seeing the rest of the EqualLogic family. EqualLogic PS Series arrays utilize a common firmware that establishes a virtualized architecture that can support up to 16 peer arrays in a Group. The entry-level EqualLogic PS4110 ships in 2.5-inch or 3.5-inch configurations that can be integrated with PS6110 arrays. The PS-M4110 is a blade array designed for the PowerEdge M1000e blade chassis that offers 14 drives and dual controllers. The FS7600/FS7610 controller adds unified NAS capability to the family.
The most significant feature that differentiates the EqualLogic family from other solutions available in the market is scale-out functionality. Dell’s EqualLogic division has engineered a family of storage devices, that aspires to scale performance linearly as overall capacity scales. In other words, rather than simply providing scale-up for storage capacity, EqualLogic scale-out technology creates an upgrade path for additional memory and controllers that can provide comparable boosts to performance.
Of course the entire EqualLogic family is about more than just hardware. For one, their software offerings allow for deep integration with VMware, , including VAAI, VASA and SRM. Dell also includes some of the most robust Linux host integration tools (HIT). Beyond that, the EqualLogic HIT meshes with Microsoft environments and key Microsoft applications, giving the platform more flexibility in a wider variety of environments. Regardless of the environment, the EqualLogic platform is designed to operate easily with intuitive access to all of the key storage functions, including more advanced features like replication, thin provisioning, snap shots and clones.
In addition to the integration tools, Dell EqualLogic also offers monitoring and diagnostics tools via SAN HQ, which gives storage admins a detailed look at what the arrays are doing at any given time. Should any issues arise, the SAN HQ logs can also be shared with Dell tech support, which gives Dell a faster path to diagnosing any problems in configuration. As much as storage is often about performance, and that's important here too, the Dell EqualLogic platform makes a strong argument about usability too; making the platform as close to "set it and forget it" as possible.
Dell EqualLogic PS6110XS Specifications
- Storage Controllers: Dual controllers, each with 4GB memory
- Storage: 13TB total capacity (7 hot-pluggable SSDs: 2.5" 400GB SSDs; 17 hot-pluggable SAS HDDs: 2.5" 600GB 10K RPM SAS)
- Supported OS: Windows Server 2000/2003/2008 and 2008 R2 (including Hyper-V), Windows Vista and XP, Red Hat Enterprise Linux, SUSE Linux Enterprise, Sun Solaris, IBM AIX, HP-UX, Mac OS X, VMware ESX Server 3 / 3.5 / 4 / 4.1 / 5, ESXi 3.5 / 4 / 4.1 / 5, Citrix XenServer, Novell NetWare
- RAID Support: RAID-6 Accelerated
- 10GbE Network Interfaces: 10GbE connectivity with 1 x 10GBASE-T with RJ45, 1 x 10GbE SFP+ for fiber or twin-ax copper cabling, and 1 x 100BASE-TX dedicated management port per controller
- TCP Network Support: IPv4, IPv6 core support
- Power supplies: Dual 700 W
- Environmental Operating Conditions
- Temperature range: 5° to 40°C (41° to 104°F)
- Storage temperature: –40° to 65°C (–40° to 149°F)
- Dimensions (HxWxD): 8.68cm (3.41") x 44.63cm (17.57") x Depth: 54.1cm (21.3")
- Weight: 25.4 kg (56 lbs)
Design and Build
The Dell EqualLogic PS6110XS is designed to blend in with existing Dell PowerEdge-equipped datacenters, leveraging a similar front bezel design and drive bay configuration. That said, if you leave the bezel off, the shiny metallic drive trays really do stand out from the crowd. The PS6110XS packs 24 2.5" SSD/HDD drive bays in the front; each individually labeled so users know which are SSDs and which are traditional HDDs. From a local management perspective, the front of the chassis is very basic, offering go/no-go LEDs to show the current status of power, network connectivity, and array errors. As with many enterprise storage platforms, power is always on when a power supply is connected, there is no separate chassis on/off button. One minor feature to make servicing the platform easier, is the right tab includes a sticker with the Dell Express Service code. This makes it easier for calling in to support without reading it off the top or rear of the chassis.
The rear of the PS6110XS houses two controller modules, each incorporating independent network connectivity and out-of-band management access. The chassis also features two independent power supplies which load-balance during use and can individually fully power the array in the event of a PSU failure.
EqualLogic PS6110XS deployments use dual storage controllers designed to operate in an active/passive configuration. When the first controller is in use, the second controller remains in a passive operating mode, with its Ethernet ports electrically inactive and activity LEDs unlit until a vertical port failover has occurred. The same goes for the network connection on the 10GbE unit, meaning that while each controller has one Twinax port and one 10Gbase-T port, only one is active on a single controller at any given time. If both Twinax and 10Gbase-t connections are present, the Twinax side has higher priority and is connected first.
Dell did an excellent job designing the chassis and controllers in the EqualLogic PS6110XS, making them easy to service and putting a great deal of attention cooling. Removing a controller shows a layout that uses few if any cables to connect hot-swap components, which instead utilize quick-connect fittings mounted directly to the circuit boards. This improves both cooling by not having excess wires disturbing airflow across components and means you can slot load modules without worrying about missing internal power or data cables.
According to Dell, approximately 89% of EqualLogic array users work in virtualized environments. The Dell EqualLogic family integrates with Microsoft, Linux and VMware applications and platforms and offers automatic management of snapshots and replication. The PS6110XS features a complete VAAI integration with VMware, a key advantage for clients with VMware environments -- whereas VMware certification only guarantees that storage devices are able to offer storage services to VMware via LUN, integration means that EqualLogic management tools are available directly from vCenter. Dell also goes to great lengths to reduce the amount of time administrators need to spend getting the array setup and integrated into a given platform. For environment such as VMware, these tools and scripts were invaluable throughout the review process.
When it comes to getting a given storage platform integrated with a virtualized environment, there is no shortage of whitepapers to give suggestions to help through that process. These are great reference points, but still leave admins doing the legwork to get the storage platform connected. The EqualLogic team takes things a step further by providing scripts that take attributes from your environment (usernames, passwords, NIC names, IP addresses, etc) and seamlessly handle the connections. For environments leveraging MPIO settings across multiple VMware hosts, these tools dramatically reduce the time needed to get the array properly configured in the environment. Below is an example of the two lines of code needed to create the iSCSI vSwitch and then activate MPIO across the available connections to the array.
- setup.pl --username=root --password=storagereview --configure --server=18.104.22.168 --vswitch=vSwitchISCSI --mtu=1500 --nics=vmnic3,vmnic4 --ips=22.214.171.124,126.96.36.199 --netmask=255.255.0.0 --vmkernel=iSCSI --nohwiscsi --enableswiscsi --groupip=188.8.131.52
- setup.pl --username=root --password=storagereview --install --server=184.108.40.206 --reboot
Dell Virtual Storage Manager 3.5 (VSM) operates as a VMware vCenter plug-in to allow administrators to manage EqualLogic-backed virtual environments from a centralized console, including management of multiple EqualLogic groups. VSM 3.5 incorporates Dell’s Datastores Manager to create, resize, and delete datastores, as well as VSM Smart Copies and Replication for hypervisor-consistent snapshots, clones, and replicas and VDI Tool for thin cloning within a VMware View environment.
The EqualLogic PS6110XS offers a comprehensive management WebGUI for both individual arrays as well as large pools populated with many arrays. Out of the box, this out-of-band interface is brought online and found by a setup utility that sets the network IP addresses as well as admin user credentials. Once set up, the WebGUI allows users to manage all aspects of each array, from creating LUNs, adjusting permissions, initiating snapshots, creating a replication partner to updating the software on the platform. The one item missing that has been taken out in favor of Dell's external SAN Headquarters (SAN HQ) is performance data.
Dell EqualLogic SAN Headquarters (SAN HQ) is an external monitoring and reporting tool that aggregates information on performance, capacity, and alerts across multiple groups of EqualLogic PS arrays. To provide enhanced support, the EqualLogic family can be configured to automatically transmit management and diagnostic information to Dell via SAN HQ SupportAssist. In our testing environment, SAN HQ was another invaluable tool to monitor disk activity during benchmarks to verify LUNs were tiered in flash or even monitoring latency in ways our benchmark doesn’t capture at the low level.
The Dell EqualLogic comes packed with features to help users protect data with point in time and synchronous replication across LAN and WAN networks for disaster recovery that can support both individual PS array failures or up to entire storage pools. Dell also ties in with VMware snapshot technology to protect individual VMs on top of intelligent snapshot technology built into the array itself to provide the best of both worlds; a scalable and automated data protection system for a virtual environment.
The StorageReview Enterprise Test Lab allows us to provide relevant and unbiased benchmarks of enterprise storage devices by establishing a testing environment comparable to what IT administrators encounter in real datacenter and field deployments. The Enterprise Test Lab incorporates a variety of servers, networking, power conditioning, and other infrastructure to design real-world configurations that accurately reflect the performance of storage devices during reviews.
We incorporate details about our lab environment and testing protocols into reviews so that storage administrators and those responsible for equipment acquisition can fairly gauge the conditions under which we have achieved the published results. None of our reviews are paid for or overseen by the manufacturer of equipment we are testing. Additional details about the StorageReview Enterprise Test Lab and an overview of its networking capabilities are available on those respective pages.
In order to fairly evaluate the performance of the Dell EqualLogic PS6110XS, we established an environment that ensures that the PS6110XS is the I/O bottleneck rather than the network. This review employs dual Lenovo ThinkServer RD630s paired with a Mellanox SX1036 10/40Gb Ethernet Switch, with connectivity via Mellanox ConnectX-3 PCIe adapters.
- 2 x Intel Xeon E5-2620 (2.0GHz, 15MB Cache, 6-cores)
- Intel C602 Chipset
- Memory - 16GB (2 x 8GB) 1333Mhz DDR3 Registered RDIMMs
- Windows Server 2008 R2 SP1 64-bit, Windows Server 2012 Standard, CentOS 6.3 64-Bit
- 100GB Micron RealSSD P400e Boot SSD
- LSI 9211-4i SAS/SATA 6.0Gb/s HBA (For boot SSDs)
- LSI 9207-8i SAS/SATA 6.0Gb/s HBA (For benchmarking SSDs or HDDs)
- Intel X540-T2 10GbE PCIe 2.1 Adapter
- Mellanox ConnectX-3 10GbE PCIe 3.0 Adapter
- Mellanox ConnectX-3 InfiniBand PCIe 3.0 Adapter
- 36 40GbE Ports (Up to 64 10GbE Ports)
- QSFP splitter cables 40GbE to 4x10GbE
- Mellanox ConnectX-3 EN PCIe 3.0 Twin 10G Ethernet Adapter
Application Performance Analysis
In the enterprise market there is a huge difference between how products claim to perform on paper and how they perform in a production environment. We understand the importance of evaluating storage as a component of larger systems, most importantly how responsive storage is when interacting with key enterprise applications. To this end, we've rolled out application tests; in this review we are introducing the VMware VMmark Virtualization Benchmark.
The VMware VMmark 2.5.1 benchmark measures the performance of a given hardware platform by running a set number of Tiles, which contain VMs performing common workloads such as mailserver, web 2.0, database and other activities. In our testing environment by keeping the networking and compute hardware static and swapping out different storage arrays, we turned it into a benchmark that allows us to measure the effect of storage in a virtualized infrastructure.
Given the heavy I/O load in VMmark 2.5.1 (about 2,500 IOPS for 1 Tile) the EqualLogic PS6110XS was able to scale well under load as long as the shared storage fit inside its flash tier. We were able to scale up to 4 Tiles with the PS6110XS (1,600GB of provisioned storage), whereas 6 Tile (2,300GB of provisioned storage) surpassed the amount that could fit completely in flash. With SAN HQ running in the background, we measured roughly 7,100 IOPS with an average transfer size of 20KB with a 30/70 read/write spread at 4 Tiles, 4,500 IOPS at 2 Tiles and 2,500 IOPS with 1 Tile. Internal load on the SSDs peaked at QD6 for each individual SSD and QD25-30 for the HDDs at 4 Tiles, whereas for 1 Tile the SSDs were at QD6 and the HDDs at QD5.
Enterprise Synthetic Workload Analysis
Our enterprise storage benchmark process preconditions each device into steady-state with the same workload the device will be tested with under a heavy load of 16 threads with an outstanding queue of 16 per thread, and then tested in set intervals in multiple thread/queue depth profiles to show performance under light and heavy usage. With the Dell EqualLogic PS6110XS we also measured the performance of two versions of SSDs Dell equipped with the hybrid array, an original set that came with our review system and a recently updated set that is shipping with new systems today.
Preconditioning and Primary Steady-State Tests:
- Throughput (Read+Write IOPS Aggregate)
- Average Latency (Read+Write Latency Averaged Together)
- Max Latency (Peak Read or Write Latency)
- Latency Standard Deviation (Read+Write Standard Deviation Averaged Together)
Our Enterprise Synthetic Workload Analysis includes four profiles based on real-world tasks. These profiles have been developed to make it easier to compare to our past benchmarks as well as widely-published values such as max 4k read and write speed and 8k 70/30, which is commonly used for enterprise systems.
- 100% Read or 100% Write
- 100% 4k
- 8k 70/30
- 70% Read, 30% Write
- 100% 8k
In the first of our enterprise workloads, we measured a long sample of random 4k performance with 100% write and 100% read activity to get our main results. During our 6 hour preconditioning period we verified the data we were measuring was fully tiered on the SSDs to show optimal performance out of the array. This included 8 50GB LUNs, split between two servers. With the PS6110XS in steady-state we measured peak random 4K throughput at 37,171 IOPS read and 8,605 IOPS write.
With a heavy effective queue depth of 512 (256QD per server) average latency from the EqualLogic PS6110XS measured 13.77ms read aggregate from both servers and 59.48ms write.
Measuring peak response times in our heavy 4K random read and write test, the EqualLogic PS6110XS remained well behaved, with max latency measuring 135-302ms read and 287-349ms write between both test servers.
Switching our focus to latency consistency, we noted a big improvement transitioning from the original SSDs to the newer SSDs in terms of lower standard deviation. Read latency improved substantially, with strong gains in write latency as well.
Compared to the fixed 16 thread, 16 queue max workload we performed in the 100% 4k write test, our mixed workload profiles scale the performance across a wide range of thread/queue combinations. In these tests, we span workload intensity from 2 threads and 2 queue up to 16 threads and 16 queue. In the expanded 8k 70/30 test, we measured performance from the EqualLogic PS6110XS scaling from 9,219 IOPS aggregate at 2T/2Q to 15,408 IOPS at 16T/16Q. This was a good bump in performance from the older style of SSD which scaled from 7,184 IOPS at 2T/2Q up to 13,591 IOPS at 16T/16Q.
Evaluating the 8k 70/30 average response times, the Dell EqualLogic PS6110XS had latency measuring 0.86ms aggregate at 2T/2Q (QD8 effective from both servers combined) and scaled up to 33.215ms at 16T/16Q (QD512 effective). This compared to the older SSDs which scaled from 1.105ms at 2T/2Q up to 37.655ms at 16T/16Q.
Comparing peak response times between the older and newer SSDs equipped in the EqualLogic PS6110XS, they were both similar across the board floating between 115-250ms, with the exception of two spikes that measured 766ms on the older SSDs and 2,180ms on the newer SSDs in the QD32-64 (EQD64-128) range.
While both SSD types had two large response time spikes in our main test sampling time, switching our view towards latency consistency showed a better picture where the PS6110XS remained calm under load up into the higher queue depth ranges. At the highest combined load, the newer SSD type was able to offer lower standard deviation than the older style of SSD.
The Dell EqualLogic PS6110XS system ships with 24 drives in a hybrid array, that top out at 13TB depending on RAID configuration. Our review unit features 17 x 600GB SFF SAS hard drives and 7 x 400GB enterprise-grade SAS SSDs, which can utilize dual storage controllers for redundancy. The key features that demarcate the EqualLogic family, including the PS6110XS, from other similar storage offerings is its scale-out capability, ease of use and deployment. None of these "ease" features are more evident and beneficial than automated data management, which encompasses automated tiering and load management. Of course the EqualLogic software offering is extremely detailed as well, Dell offers integration tools for virtualized and Linux environments, as well as specific plugins for popular Microsoft software packages.
When looking at the SMB/midmarket storage market at large, it's important to understand that the EqualLogic family is not designed to compete on price versus smaller players and startups, offering high-capacity NAS and JBOD solutions. It's relatively simple to engineer a storage system where administrators can add a JBOD array and offer scale-up capacities. EqualLogic provides the versatility to offer comparable increases in performance by scaling out memory and processing across multiple arrays. Dell EqualLogic is also oriented toward supporting customers that use their peer scaling model and automatic tiering functionality. This keeps storage arrays in use through generational upgrades that often force SAN administrators to retire arrays from competing platforms, when they can no longer serve as front-line storage. This is an approach that offers a lot of promise, in an era where growing capacity requirements is one of the only certainties for storage architects and administrators.
Ultimately, being easy to use isn't good enough in the enterprise. The array has to be able to support the varied and irregular workloads it's sure to see. When we look at a hybrid array like this, we want to see clear indications that flash is being used efficiently and is properly accelerating workloads. This requires a few elements that Dell hit the nail on the head with, an area which quite a few hybrid array vendors miss entirely with their goal of maximizing their IOPS/dollar potential. The EqualLogic PS6110XS mixes 7 400GB SSDs in a 2TB RAID6 pool, with 17 600GB 10K SAS HDDS in a 9TB RAID6 pool. Accounting for RAID overhead, that gives the PS6110XS a HDD/flash ratio of 4.5:1, compared to 10:1 at best down to 30:1 at worst from some hybrid platforms on the market. With the PS6110XS, Dell aims to lower the I/O cliff for data that can't fit inside flash; falling from SSD to 10K HDD, versus SSD to 7K HDD that many hybrid solutions offer. The Dell EqualLogic PS6110XS ends up being far more balanced for large intensive virtualized workloads, which we showed in our VMmark test that peaked with 1.6TB of I/O intensive storage. The downside is the upfront cost, although long term you aren't upgrading the SSDs or tearing out the platform for something larger as your demands grow. At a certain point it's about buying the right solution for serious storage needs, not the cheapest per GB or IOP.
- Sophisticated VMware integration places EqualLogic management directly into vCenter
- Scales-out to improve performance along with increased capacity
- Smartly designed 4.5:1 HDD/Flash storage ratio that prevents a dreaded I/O cliff scenario
- Impressive Dell support offerings
- Higher per-GB cost for storage than some other choices for mass storage
The Dell EqualLogic PS6110XS is versatile, easy to administrate, and engineered to be used through more than one generation of storage technology. For SMBs and midmarket buyers who need their storage to be flexible and can take advantage of Dell's wide application and OS support via host integration tools, the PS6110XS is an industry leading solution.