February 21st, 2014 by StorageReview Enterprise Lab
Toshiba PX02SM Series SAS3 Enterprise SSD Review
The Toshiba PX02SM is an enterprise SSD which utilizes a dual port 12Gb/s SAS interface and eMLC NAND and achieved promising results in our early review of the drive. The PX02SM was the first 12Gb/s SAS SSD on the market, predating access to SAS3 adapters that could allow the StorageReview Enterprise Test Lab to measure its maximum performance with sequential workloads. With the addition of a Supermicro SuperStorage Server AR24NV SAS3 platform to the lab, we can revisit the PX02SM and put it through its paces without HBA bottlenecks from the host or custom configurations.
The PX02SM is part of Toshiba’s PX-series which debuted in 2013 with faster SAS3 interfaces. The PX02SM offers the fastest specifications, layered ECC for enhanced error correction, and is available in security models that use cryptographic erase for improved security. The Toshiba PX02SM is available in 200GB, 400GB, 800GB and 1.6TB capacities backed by a five-year limited warranty. Our review will benchmark both the 400GB and 800GB models and provide 6Gb/s SAS2 results for comparison.
Toshiba PX02SM Specifications
- 200GB (PX02SMF020, PX02SMU020)
- 400GB (PX02SMF040, PX02SMU040)
- 800GB (PX02SMF080, PX02SMU080)
- 1.6TB (PX02SMB160, PX02SMQ160)
- NAND: 24nm eMLC
- Interface: SAS 6Gb/s and 12Gb/s
- Form Factor: 2.5” SFF x 7mm High (15mm for 1.6TB model)
- Sequential Read (Sustained): 900MB/s
- Sequential Write (Sustained): 400MB/s
- Random Read 4k (IOPS): 120,000
- Random Write 4k (IOPS): 30,000
- Temp - Operating 0° to 55°C
- Temp - Non-Operating -40° to 70°C (-40° to 158°F)
- Vibration - Operating 9.8 m/s² (1G)
- Vibration - Non-Operating 49 m/s² (5G)
- Shock - Operating 9,800 m/s² (1,000G 0.5ms, ½ sine)
- Shock - Non-Operating 9,800 m/s² (1,000G 0.5ms, ½ sine)
- Power Loss Protection
- Endurance TBW: 3.7PB (200GB), 7.3PB (400GB), 14.6PB (800GB), 29.2PB (1.6TB)
- MTTF: 2 Million Hours
- Dimensions (WxDxH): 69.85 mm x 100 mm x 7.0 mm
- Weight: 70g
- 5-year Limited Warranty
Design and Build
While high-performance enterprise SSDs generally use a 15mm chassis, the Toshiba PX02SM uses a 7mm design (except for the 1.6TB model, which is 15mm thick) and in a 2.5-inch form factor. The drive's exterior is functional, not flashy, and it feels sturdy with a stamped aluminum body.
On the front of the PX02SM SSD is the 12Gb/s SAS connection for power and data, which is backwards compatible with SAS 6Gb/s HBAs.
The PX02SM makes use of a Marvell co-branded TC58NC9036GTC controller. Our 400GB review model also has 16 of Toshiba's 24nm eMLC NAND die packages, which each have a capacity of 32GB o512GB of raw flash and an unformatted capacity of 400GB.
Testing Background and Comparables
The Toshiba PX02SM uses a Marvell co-branded TC58NC9036GTC controller and 24nm eMLC NAND with an interface that supports SAS 12Gb/s. While our protocol will also publish the performance results of the PX02SM 400GB model with a 6Gb/s SAS2 host, the best performance for the 400GB and 800GB models is achieved via SAS3. The StorageReview Enterprise Test Lab uses a SuperMicro SuperStorage Server 2027R-AR24NV as our SAS3 testbed, featuring:
- 2 x Intel Xeon E5-2687 v2 (3.4GHz, 25MB Cache, 8-cores)
- Intel C602 Chipset
- Memory - 256GB (16 x 16GB) 1333Mhz Micron DDR3 Registered RDIMMs
- Windows Server 2012 Standard - 100GB Micron RealSSD P400e Boot SSD
- 3 x Supermicro SAS3 HBAs (LSI SAS 3008 Controllers)
- 100GB Micron P400e Linux CentOS 6.3 boot
- 200GB Micron P400m Windows Server 2012 boot
- 100GB Micron P400e Linux CentOS 6.3 boot (Sysbench) w/ Micron M500 960GB for Database storage
- Mellanox ConnectX-3 Dual-Port VPI PCIe 3.0 Adapter
Comparables for this review:
- Hitachi SSD800MH (400GB, Intel co-branded DB29AA11B0 controller, Intel 25nm MLC NAND, 12.0Gb/s SAS)
- OCZ Talos 2 R (400GB, SandForce SF-2500 controller, Intel 25nm MLC NAND, 6.0Gb/s SAS)
- Hitachi SSD400M (400GB, Intel EW29AA31AA1 controller, Intel 25nm eMLC NAND, 6.0Gb/s SAS)
- Smart Optimus (400GB, third-party controller, Toshiba 34nm MLC NAND, 6.0Gb/s SAS)
- STEC s842 (s840 series) (800GB, STEC 24950-15555-XC1 controller, Toshiba MLC NAND, 6.0Gb/s SAS)
Application Performance Analysis
In order to understand the performance characteristics of enterprise storage devices, it is essential to model the infrastructure and the application workloads commonly found in live production environments. Our first two benchmarks of the Toshiba PX02SM are therefore a MarkLogic NoSQL Database Storage Benchmark and MySQL performance via SysBench.
Our MarkLogic NoSQL Database environment requires groups of four SSDs with a usable capacity of at least 200GB, since the NoSQL database requires roughly 650GB of space for its four database nodes. Our protocol uses an SCST host and presents each SSD in JBOD, with one allocated per database node. The test repeats itself over 24 intervals, requiring between 30-36 hours total for the SSDs in this class. MarkLogic records total average latency as well as interval latency for each SSD.
The Toshiba PX02SM had the highest average latency of any of the comparable drives in the MarkLogic NoSQL benchmark at 3.604ms, a notably poor result in this test.
The PX02SM architecture is not optimized for the access patterns of our NoSQL workload, triggering latencies that spiked at or above the normalized maximum value of 9ms throughout the protocol.
The HGST SSD800MM maintained a decisive lead over the comparables, with only occasional spikes during merge read and merge write operations.
The SanDisk Optimus kept latency for most operations below 6ms, with a handful of spikes reaching between 7ms and 11ms.
The HGST SSD400M had greater variation in latency than the SanDisk Optimus, with the greatest latencies during NoSQL journal write operations.
The OCZ Talos 2 R had similar NoSQL performance to the SSD400M, with latency peaks between 9-32ms. Its highest peaks occurred during merge write operations.
Our next application test consists of Percona MySQL database test via SysBench, which measures the performance of OLTP activity. In this testing configuration, we use a group of Lenovo ThinkServer RD630s and load a database environment onto a single drive. This test measures average TPS (Transactions Per Second), average latency, as well as average 99th percentile latency over a range of 2 to 32 threads. Percona and MariaDB are using the Fusion-io flash-aware application APIs in the most recent releases of their databases, although for the purposes of this comparison we test each device in their "legacy" block-storage modes.
The 800GB PX02SM performed better than all SAS2 comparables with 1,755 transactions per second at 32 threads, but behind the Hitachi SSD800MM which hit the top MySQL TPS value of 2,113 TPS.
Average latency results during the MySQL application benchmark fall along similar lines. The 800GB Toshiba PX02SM places second to the Hitachi SSD800MM, with average latencies that range from 6.63ms with 2 threads to 18.23ms with 32 threads.
Moving to our test of worst-case MySQL latency with the OLTP workload, the PX02SM holds its own across the range of thread counts but falls within the middle of the pack of comparables.
Enterprise Synthetic Workload Analysis
Flash performance varies as the drive becomes conditioned to its workload, meaning that flash storage must be preconditioned before each of the fio synthetic benchmarks in order to ensure that the benchmarks are accurate. Each of the comparable drives are secure erased using the vendor's tools and preconditioned into steady-state with a heavy load of 16 threads and an outstanding queue of 16 per thread.
Preconditioning and Primary Steady-State Tests:
- Throughput (Read+Write IOPS Aggregate)
- Average Latency (Read+Write Latency Averaged Together)
- Max Latency (Peak Read or Write Latency)
- Latency Standard Deviation (Read+Write Standard Deviation Averaged Together)
Once preconditioning is complete, each device is then tested in intervals across multiple thread/queue depth profiles to show performance under light and heavy usage. Our synthetic workload analysis for the Toshiba PX02SM uses two profiles which are widely used in manufacturer specifications and benchmarks.
- 100% Read and 100% Write
- 70% Read/30% Write
While the PX02SM was not able to match the performance of HGST's SSD800MM during 4k preconditioning, both PX02SM capacities were able to hit 50,000 IOPS burst speed before tapering to approximately 29,000 IOPS at steady state.
With a load of 16 threads and a queue depth of 16, the PX02SM hit burst latencies of approximately 5ms before leveling off near 8.7ms, behind only the HGST comparables.
Maximum latencies measured during 4k preconditioning continued the same pattern: the Toshiba PX02SM is able to maintain a consistent lead over the competition except for the HGST SSD800MM and SSD400M which kept the lowest maximum values.
Standard deviation results make it easier to see how consistent the latencies were throughout the preconditioning process. These results did not reveal any surprises, with both of the PX02SM drives begining the process near 2.4ms and reaching steady state with a latency standard deviation near 5.1ms for the 800GB model and 5.8ms for the 400GB model.
Having competed the preconditioning process, the HGST SSD800MM took the lead 4k throughput position. The Toshiba PX02SM also had a strong performance with our SAS3 testbed, particularly for 4k read operations where it achieved 109,135 IOPS in its 400GB form and 108,633 IOPS as an 800GB drive.
While the SSD800MM commands the lowest average 4k latencies in read and write operations, the PX02SM demonstrates a profiency for small read operations, keeping average latencies to 2.34ms and 2.35ms in the SAS3 testbed. The 4k average latency results also reveal the difference the host and HBA make, with slightly lower read latencies via 6Gb/s SAS2 than 12Gb/s SAS3.
The maximum latencies measured during our 4k synthetic benchmark place the Toshiba PX02SM near the middle of the pack of comparables.
The 4k standard deviation calculations reflect the Toshiba PX02SM's consistent latency performance during the test, with results that place the PX02SM ahead of all the competition except the SSD800MM regardless of whether it was benchmarked with our 6Gb/s SAS or 12Gb/s host.
Our next workload uses 8k transfers with a ratio of 70% read operations and 30% write operations. The PX02SM began preconditioning for this benchmark with a second-place burst of approximately 75,000 IOPS before settling into steady state near 41,000 IOPS for the 400GB model and 43,000 IOPS for the 800GB model.
The Toshiba PX02SM began 8k 70/30 preconditioning with an average latency of 3.42ms which increased to 6.22ms for our 400GB review drive and 5.96ms for our 800GB drive, results placing it behind the best-in-class HGST SSD800MM and near the STEC s842.
With only a handful of spikes, the PX02SM continues to maintain strong maximum latency results throughout preconditioning, winding up near 36ms for the 400GB drive and 40ms for the 800GB model.
At the beginning of the preconditioning curve, the PX02SM -- both capacities and SAS generations -- is able to achieve more consistent latencies than the SSD800MM when the latter is accessed via SAS2, but this advantage disappears by the end of preconditioning when the PX02SM takes its customary second-place finish in the standard deviation results.
Once the drives are preconditioned, the 8k 70/30 throughput benchmark varies workload intensity from 2 threads and 2 queue up to 16 threads and 16 queue. Measuring 8k throughput, the PX02SM did not see much benefit from the shift to 12Gb/s SAS but maintained a strong second-place showing regardless. With the maximum workload, the 400GB PX02SM reached 40,887 IOPS and the 800GB model hit 42,589 IOPS.
AAverage latencies in the 8k 70/30 test remained higher than the HGST SSD800MM comparables and nearly neck-and-neck with the STEC s842.
At lower thread counts and queue depths, the maximum recorded latencies during our 8k synthetic benchmark make it difficult to distinguish between the Toshiba PX02SM, the HGST SSD800MM, and the STEC s842. At higher thread counts the SSD800MM regains its decisive lead and the PX02SM pulls away from the STEC s842.
Standard deviation plots for latency during the 8k 70/30 test do not reveal any weak points in PX02SM performance. The PX02SM 12Gb/s results scale from 0.27ms with 2 threads and a queue depth of 2, up to approximately 3.7ms with the most intensive workloads.
The PX02SM is one Toshiba's new high performance enterprise SSDs, with 12Gb/s SAS connectivity, eMLC NAND, and a range of capacities up to 1.6TB that help make it suitable to a broad range of applications. Except for the 1.6TB edition, the PX02SM is also notable for its 7mm height which also opens doors and slots for the drive. Toshiba’s PX02SM architecture also offers some useful features such as power-loss protection to protect in-flight data.
The PX02SM consistently outperformed most of the comparable eMLC SSDs in synthetic benchmarks, with steady-state figures that were often competitive with the burst speeds of other eMLC drives (with the exception of the HGST SSD800MM). In our MarkLogic NoSQL benchmark however, the drive performed at the bottom of the pack among the comparables for all operations. The PX02SM redeemed itself to some extent during a test of OLTP performance in MySQL, with results that didn’t equal its synthetic benchmark success but did maintain a strong second place performance to the SSD800MM.
- Strong results in synthetic benchmarks and MySQL performance
- 7mm drive height for most capacities
- Notably low performance in the NoSQL benchmark
The Bottom Line
The PX02SM is a versatile enterprise SSD that reflects the strengths of Toshiba's eMLC engineering. It performs in synthetic tests and in the MySQL environment despite a poor showing with NoSQL.