StorageReview Enterprise Test Lab
At StorageReview we know enterprise storage solutions need to be evaluated in an enterprise environment. We're often asked about not only our testing methodologies for storage products but about our environment as well. Reviews of single drives can be done just about anywhere; high-end storage solutions and arrays need a more robust testing environment. To that end, we've developed an on-site enterprise test lab to not only better simulate real world conditions, but to also offer proper compute, switching, electrical conditioning and so on. We want to give our readers a glimpse into the heart of the StorageReview Enterprise Test Lab, which is home to servers, SAN and NAS appliances, PCIe flash solutions, high-speed networking gear, and power conditioning products that make it possible to test the latest enterprise hardware from single hard drives up to the fastest flash storage arrays.
The StorageReview Enterprise Test Lab includes the following components broken down into categories:
- Rack Hardware
- Power / Temperature Monitoring
- TrippLite SRCOOL12K (12,000 BTU)
- Netgear ProSecure UTM150 4 x WAN Firewall
- Netgear ProSafe Plus XS708E 8-Port 10GbE Switch
- Netgear ProSafe GSM7352S 48-Port 1GbE Switch
- Netgear ProSafe GS752TS 48-Port 1GbE Switch
- Netgear ProSafe GS724T 24-Port 1GbE Switch
- Netgear ProSafe Plus JGS524E 24-Port 1GbE Switch
- Mellanox SX1036 36-Port 40GigE Switch
- Mellanox SX6036 36-Port 56Gb/s InfiniBand Switch
- QLogic SB5800V 8Gb Fibre Channel Switch
- Brocade 6510 16Gb Fibre Channel Switch
- Dell PowerEdge R720 G12 (x2)
- EchoStreams GridStreams High‐Density Quad‐Node Computing Platform
- HP ProLiant DL380p Gen8 (x2)
- Lenovo ThinkServer RD630 (x13)
- Lenovo ThinkServer RD530 (x2)
- Lenovo ThinkServer RD240 (x4)
- Lenovo ThinkStation D30
- Super Micro Computer SuperServer 5037A-i
- Super Micro Computer SuperServer 6047R-TXRF
- Supplementary Consumer Testing Platforms
- Enterprise Testing Software
- FIO - flexible I/O tester
- MarkLogic NoSQL Database Storage Benchmark
- Storage (NAS/SAN/JBOD)
- Protective Storage
The largest components of the StorageReview Lab are our two Eaton S-Series racks. They allow us to put storage hardware in its native environment and have it located in close proximity to power, networking, and compute hardware. When looking at environmental needs, installing hardware in a rack-mount environment also helps with proper airflow, keeping system and component temperatures down.
At the component level power consumption is continuously dropping as each new generation of product comes out, but at the datacenter level the total power required to run multiple switches, servers, SANs, and other required gear adds up to values well above what normal 120V circuits can handle. The StorageReview Lab offers a wide range of utility options, including single and three-phase power, as well as 120V circuits for smaller equipment. In total we have a power ceiling of 72,000 watts in our office, giving us plenty of headroom no matter what scale of projects we need to tackle.
To handle this massive power distribution on an outlet-by-outlet basis, we use an Eaton Advanced Monitored ePDU, Eaton 5PX and 9PX UPS, and multiple PDUs. Power is fed in through either our primary Eaton UPS with servers connected directly to it or we through our Eaton ePDU which supplies power to both of our Eaton racks. For the second rack we use auxiliary PDUs branched off the main ePDU to supply power locally.
With constant 24/7 power monitoring, we make sure tests are running smoothly, power conditions are clean, and abnormal power conditions don't affect our tests. With evaluations that take weeks or months and tests that span days individually, making sure everything is running without a blip is an important consideration when we review enterprise hardware.
Our lab operates through multiple online backup batteries, providing clean and reliable power no matter the circumstances. Our primary Eaton 9PX UPS has a capacity of 6,000VA, offering 5 to 45 minutes of uptime for the bulk of the lab. For critical equipment and one or two servers working on long-term product reviews, our lab includes an Eaton 5PX UPS with an extended battery to provide between 1 to 2.5 hours of runtime in the event of an extended power outage. As we expand our lab's capabilities we are also evaluating additional power gear to incorporate into the StorageReview Test Lab.
On the networking side, our lab equipped with the latest networking hardware, supplied from Netgear, Mellanox and QLogic. For 1GbE we use Netgear Prosafe switches, which give us plenty of connectivity ports when we evaluate products designed to work with in a small or medium business environment. For the SANs and other high-speed storage arrays, we also incorporate Mellanox and QLogic switching in our lab. For high-speed 10GbE Ethernet we use a Mellanox SX1036 10/40Gb switch equipped with fan-out cables to attach four SFP+ cables to a single QSFP port. For new compute and storage platforms supporting 10GBase-t, our rack also includes a Netgear ProSafe Plus XS708E. When it comes to supporting Fibre Channel equipment, our lab includes QLogic 8Gb switching and 16Gb QLogic and Emulex HBAs.
For solutions that demand even greater throughput and lower latency, we also have a 56Gb/s Mellanox InfiniBand fabric used for primarily testing flash appliances as well as acting as the backbone of new application tests such as our MarkLogic NoSQL Database Benchmark.
For our main compute needs, we have a cluster of Lenovo ThinkServer RD630s, RD530s, and RD240s (Romley and Westmere) in our test lab. Depending on the hardware being tested, we configure these servers for PCIe flash product reviews, enterprise hard drive and SSD reviews using LSI 9207/9211 HBAs or network device testing with Mellanox 10/40GbE or InfiniBand adapters as well as onboard Intel 1GbE NICs.
As our scope of coverage of enterprise storage products expands, we are continuously adding to our pool of compute hardware with gear from a variety of manufacturers. Having the right hardware on hand, or in some cases enough hardware, gives us breathing room to evaluate multiple products simultaneously or handle large-scale reviews that require a large computing cluster.
Outside of our primary Lenovo compute cluster, we also work with manufacturers such as HP, Dell, and Super Micro Computer with their newest Romley servers as we evaluate new testing platforms or approach product reviews that are unique to a specific manufacturer's hardware. With a growing lab of workstation and rack-mount compute hardware, we are equipped to handle nearly all enterprise storage products.
As the lab grows and evolves we'll be sure to update this post with the latest gear and more detail on how we're using it.