December 21st, 2015 by StorageReview Enterprise Lab
Memblaze PBlaze4 AIC NVMe SSD Review
Memblaze has a new generation of NVMe SSDs designed for the enterprise. The PBlaze4 is intended for data centers running cloud computing, database, and other demanding applications. It is available in both half-height, half-length (HHHL) and full-height, half-length (FHHL) form factors, and capacities ranging from 800GB to 6.4TB.
The PBlaze4 series can maintain the integrity of data being transferred in the event of power loss, with data in temporary buffers saved in NAND media. These SSDs also offer thermal throttling. The PBlaze4 series has a Pseudo-SLC (pSLC) memory management mode that's been engineered to allow an MLC to emulate the speed and durability of an SLC. PBlaze4 drives designate the portions of the memory used for metadata as pSLC for greater metadata protection and reliability. They also offer MemSpeed and MemSolid, which are designed to improve PCIe SSD performance, reliability, and Quality of Service.
Our review is of the 3.2TB HHHL version.
Memblaze PBlaze4 Specifications
- Capacity: 800GB, 1.2TB, 1.6TB, 2.4TB, 3.2TB, 6.4TB
- Sequential Read (128kb): 2.2 GB/s, 2.8 GB/s, 2.8 GB/s, 2.8 GB/s, 2.8 GB/s, 3.4 GB/s
- Sequential Write (128kb) IOPS: 700 MB/s, 1.4 GB/s, 1.4 GB/s, 2.2 GB/s, 2.2 GB/s, 2.5 GB/s
- Sustained Random Read (4kb) IOPS: 600k, 740k, 750k, 730k, 740k, 800k
- Sustained Random Write (4kb) IOPS (100% span): 60k, 240k, 150k, 320k, 200k, 250k
- Lifetime Endurance (Drive Wipes per Day): 3, 4, 3, 4, 3, 3
- Latency Read/Write: 90μs/20μs
- Uncorrectable Bit Error Rate: < 1 sector per 10^17 bits read
- Mean Time Between Failures: 2 million hours
- Form Factor: 2.5” HHHL (FHHL for the 6.4TB version)
- Interface: PCIe 3.0 x 4 (PCIe 3.0 x 8 for the 6.4TB version)
- Protocol: NVMe
- NAND Flash Memory: MLC
- Operation System: RHEL, SLES, CentOS, Ubuntu, Windows Server, VMware ESXi
- Power Consumption: <25w (<35w for the 6.4TB version)
- Operating Temperature:
- AIC: 0 – 55℃ ambient temperature with suggested airflow
- 2.5’’: 0–35℃ ambient temperature with suggested airflow, 0-70℃ case temperature
- Airflow (LFM): 300@25℃ (450@25℃ for the 6.4TB version)
- Software Support: CLI Management Tool, OS in-box driver
Design and Build
Memblaze offers their PBlaze4 in two form factors including 2.5” SSD (PBlaze4 D750) and, in this case, a half-height half-length add-in-card form factor (PBlaze4 C750). On one side it has a silver heat sink with a dark gray cover with Memblaze branding on it. The other side is an exposed circuit board where one can see the Toshiba 15nm NAND.
The PBlaze4 uses a PCIe Gen 3 x 4 connector, while the FHHL 6.4TB version uses PCIe Gen 3 x 8.
Testing Background and Comparables
The StorageReview Enterprise Test Lab provides a flexible architecture for conducting benchmarks of enterprise storage devices in an environment comparable to what administrators encounter in real deployments. The Enterprise Test Lab incorporates a variety of servers, networking, power conditioning, and other network infrastructure that allows our staff to establish real-world conditions to accurately gauge performance during our reviews.
We incorporate these details about the lab environment and protocols into reviews so that IT professionals and those responsible for storage acquisition can understand the conditions under which we have achieved the following results. None of our reviews are paid for or overseen by the manufacturer of equipment we are testing. Additional details about the StorageReview Enterprise Test Lab and an overview of its networking capabilities are available on those respective pages.
We tested the Memblaze while comparing it to the following other SSDs:
- Intel HHHL P3608 1.6TB
- Huawei HHHL ES3000v2 1.6TB
- SanDisk SHHL PX600 2.6TB
- Huawei HHHL ES3000v2 3.2TB
- HGST HHHL SN100 3.2 TB
Application Workload Analysis
In order to understand the performance characteristics of enterprise storage devices, it is essential to model the infrastructure and the application workloads found in live production environments. Our first benchmarks for the Memblae PBlaze4 are therefore the MySQL OLTP performance via SysBench and Microsoft SQL Server OLTP performance with a simulated TCP-C workload. For our application workloads each drive will be running 2-4 identically configured VMs.
StorageReview’s Microsoft SQL Server OLTP testing protocol employs the current draft of the Transaction Processing Performance Council’s Benchmark C (TPC-C), an online transaction processing benchmark that simulates the activities found in complex application environments. The TPC-C benchmark comes closer than synthetic performance benchmarks to gauging the performance strengths and bottlenecks of storage infrastructure in database environments. Each instance of our SQL Server VM for this review uses a 333GB (1,500 scale) SQL Server database and measures the transactional performance and latency under a load of 15,000 virtual users.
When looking at SQL Server Output, the Memblaze drive had a top TPS of 3,157.235 with an aggregate of 3,157.112 TPS. The best performance came from the Huawei ES3000v2, which recorded an aggregate of 3,157.34 TPS.
Looking at average latency during the 15k virtual user SQL Server benchmark test, the PBlaze4's aggregate of 7.5ms put it just behind the SanDisk and Huawei SSDs, both of which posted 7.0ms.
The next application benchmark consists of a Percona MySQL OLTP database measured via SysBench. This test measures average TPS (Transactions Per Second), average latency, as well as average 99th percentile latency. Percona and MariaDB are using the Fusion-io flash-aware application APIs in the most recent releases of their databases, although for the purposes of this comparison we test each device in their "legacy" block-storage modes.
In the average transactions per second benchmark, the Memblaze aggregate was higher than all its rivals. The PBlaze4’s top performance of a single VM was 1,509.3 TPS.
When looking at our average latency results, the Memblaze didn't do quite as well as the Intel P3608, with individual VMs running between 21.20ms through 21.66ms. However the PBlaze4 did have the lowest aggregate score of 21.50ms.
In our MySQL 99th percentile latency, a worst-case scenario, the Memblaze Performed in the middle of the pack with VMs running between 52.29ms and 52.65ms, while the top performing SanDisk PX600 drive boasted a very impressive aggregate of just 41.92ms.
Enterprise Synthetic Workload Analysis
Flash performance varies as the drive becomes conditioned to its workload, meaning that flash storage must be preconditioned before each of the fio synthetic benchmarks in order to ensure that the benchmarks are accurate. Each of the comparable drives are secure erased using the vendor's tools and preconditioned into steady-state with a heavy load of 16 threads and an outstanding queue of 16 per thread.
- Preconditioning and Primary Steady-State Tests:
- Throughput (Read+Write IOPS Aggregate)
- Average Latency (Read+Write Latency Averaged Together)
- Max Latency (Peak Read or Write Latency)
- Latency Standard Deviation (Read+Write Standard Deviation Averaged Together)
Once preconditioning is complete, each device is then tested in intervals across multiple thread/queue depth profiles to show performance under light and heavy usage. Our synthetic workload analysis for the Memblaze PBlaze4 uses two profiles, which are widely used in manufacturer specifications and benchmarks. It is important to take into consideration that synthetic workloads will never 100% represent the activity seen in production workloads, and in some ways inaccurately portray a drive in scenarios that wouldn't occur in the real world.
- 100% Read and 100% Write
- 70% Read/30% Write
In our throughput 4k write preconditioning test, the Memblaze gets points for consistency, though it did spike a bit at the beginning, starting at about 290,000 IOPS and hitting a steady-state at around 180,000. By comparison, the Huawei, HGST, and Intel drives had much more significant beginning spikes. The Huawei ES300v2 3.2TB had the highest steady-state throughput.
When testing average latency, the Memblaze was generally consistent but not outstandingly so, with a starting score around 0.8ms and reaching a steady-state around 1.4ms. This was lower than several of its competitors, but not the Intel P3608.
The PBlaze4 had the highest maximum latency of any of the five drives included in this test, plus it showed frequent spikes throughout the test. The HGST SN100 was generally more consistent, but had occasional very severe spikes, while the Intel's and Huawei's drives were very stable.Standard deviation calculations are intended make it easier visualize the consistency of the SSD latency performance results. In this test, the Memblaze offering started out with the highest latency, though it eventually ended up doing better than the Huawei ES3000v2 1.6TB. Its steadystate latency was higher than the other three drives, however.
Testing for 4k throughput, the Memblaze drive was one of the slower performers with 705,868 IOPS when reading and 180,006 writing. For comparison, the Intel drive showed the best write performance with an impressive 851,693 IOPS.
The PBlaze4 came out in the middle of the pack in average latency with 0.36ms read and 1.42 write. It was solidly beaten by the Intel P3608 when reading and the Huawei ES3000v2 3.2TB in writing.
In maximum latency, the Memblaze offering had the best read score at 4.6ms, but it's 179.8ms write speed was by far the worst.
The standard deviation scores again had the PBlaze4 on top in reading at 0.107ms, but this time it wasn't near the bottom when writing, which that "honor" going the Huawei ES3000v2 1.6TB
Our next workload uses 8k transfers with a ratio of 70% read operations and 30% write operations. Again, we will start off with the preconditioning results before switching to the main tests.
In throughput, the Memblaze offering had remarkably consistent scores, staying around 170,000 IOPS during the entire test run, while its comparables showed much more variation. The PBlaze4 again ended up in the middle of the pack once all had hit a steady-state.
Our average latency test repeated the same story, with the Memblaze drive hovering between 1.4ms and 1.5ms the entire test, while its rivals had much more variation. Once all had hit a steady-state, though, the Huawei ES3000v2 3.2TB was the clear winner.
The PBlaze4 saw spikes throughout the max latency results, though these weren't as bad as the ones occasionally experienced by the HGST SN100. Huawei's SSDs were more stable, and Intel's was the most stable of all, staying around the 15ms latency mark throughout the entire test.
In standard deviation preconditioning we again see the PBlaze4 keep a fairly consistent score throughout our testing coming in near the middle of the pack overall.
After fully preconditioning the Memblaze drive, we put it through our main 8k 70/30 test.
In throughput, this drive stayed solidly in the middle of the pack, competing neck-and-neck with the Intel, HGST, and 1.6TB Huawei offerings. The Huawei ES3000v2 3.2TB was the clear winner, however, outperforming its rivals in all conditions.
Average latency mirrored the throughput results, with the Memblaze drive competing strongly with the Intel, HGST, and 1.6TB Huawei one's, and the ES3000v2 3.2TB again taking the lead.
The story was different with maximum latency, however, with the PBlaze4 running noticeably higher latency than its comparables. The other four drives all performed much better, with Huawei's offerings showing the best performance.
Our standard deviation results show that the Memblaze drive again performing about the same as the Intel and HGST ones across a range of conditions, but as conditions became more demanding the 1.6GB Huawei drive began to show better performance while the 3.2GB drive started to do worse.
The Memblaze PBlaze4 is an enterprise-grade SSD, with the 6.4TB FHHL AIC version being the company's flagship project. The PBlaze4 has NVMe over PCIe interface and it is offered in both HHHL and FHHL (as well as 2.5") and rungs from 800GB to 6.4TB in capacity. We tested the 3.2TB HHHL version in this review, whereas the 2.5" version we reviewed previously ranked quite well and closely matched the performance profile of its AIC brother. Being offered in various form factors gives customers flexibility when chosing which one might best fit a given platform. If hot-swappability or integration into a newer server that offers front-mount access is possible, the 2.5" models offer some advantages. If the server doesn't support 2.5" NVMe SSDs, AICs are favorable being compatible with nearly all modern servers.
The Memblaze PBlaze4 AIC showed itself to be a good SSD in our benchmarks, although had some spots where it could be improved it did especially well in some of our database tests, with Sysbench giving it an aggregate score of 5954.8 TPS, beating a number of similar SSDs. The Memblaze drive stayed in the middle of the pack in our 8K 70% Read/30% write test as well as in our 4K 100% Read test measuring 705,868 IOPS. Similar to the performance profile we noted when reviewing the 2.5" PBlaze4, the AIC had some max latency spikes pop up under higher loads. Memblaze has been quick to react to this though and has just released an update that has lowered these values back into normal ranges for the NVMe SSD group.
- Strong database performance
- Solid performer in most situations
- Flexible design with a broad capacity offering
- Showed some higher max latency spikes
The Memblaze PBlaze4 is an enterprise grade SSD AIC that shows very good performance in database conditions, and generally average performance in other situations.