March 3rd, 2014 by Josh Linden
Toshiba PX02SS Enterprise SSD Review
The Toshiba PX02SS is a 2.5-inch enterprise SSD designed for transaction-intensive workloads including OLTP database applications and features a dual-port 12Gb/s SAS3 interface. The PX02SS and is available in 100GB, 200GB, 400GB, and 800GB capacities and offers notable write endurance among its class due to an overprovisioned Toshiba 24nm eMLC architecture. The Toshiba PX02SS is specified for 5.48PB total bytes written for the 100GB edition up to 43.8PB TBW for the 800GB SSD.
In 2013, Toshiba debuted a range of new SSD offerings and then inaugurated 2014 by completing its acquisition of OCZ Technology, one of several manufacturers which use Toshiba’s NAND memory in their drives. The pattern is clear: Toshiba is increasing its reach as an SSD manufacturer in its own right. Along with the Toshiba PX02SM which we also recently reviewed, the PX02SS is part of Toshiba’s new PX-series of 12Gb/s SAS3 enterprise SSDs which shares some functionality and design across the series.
Like the PX02SM, the PX02SS is available as a standard drive or with self-encrypting technology, and features Toshiba’s quadruple swing by code layered ECC error correction and power loss protection. The core differences between the two are largely endurance related; the SS is simply designed for more write intensive workloads. At the top end 800GB capacity, the SS can handle nearly 44 TBW. The SM on the other hand gives up endurance and is designed for more balanced workloads and can top out at 1.6TB capacity as a result. Both SSDs carry a five year warranty.
Our review of the PX02SS includes four 400GB capacity drives.
Toshiba PX02SS Specifications
- Capacities (Standard Model, Security Model)
- 100GB (PX02SSF010, PX02SSU010)
- 200GB (PX02SSF020, PX02SSU020)
- 400GB (PX02SSF040, PX02SSU040)
- 800GB (PX02SSB080, PX02SSQ080)
- NAND Technology: 24nm eMLC
- Drive Interface: SAS 12 Gb/sec, Dual port
- Sector Size: 512B, 520B, 528B, 4096B, 4104B, 4160B and 4224B
- 4KiB Random Read (Sustained): 130,000 IOPS
- 4KiB Random Write (Sustained): 42,000 IOPS
- 4KiB Random 70/30 Read/Write(Sustained): 80,000 IOPS
- 64KiB Sequential Read(Sustained):
- 100GB, 200GB, 400GB: 1,100MiB/s (Standard Model), 910MiB/s (Security Model)
- 800GB: 1,060MiB/s (Standard Model), 910MiB/s (Security Model)
- 64KiB Sequential Write (Sustained): 410 MiB/s
- Voltage: 5V (+/- 5%), 12V (+/- 5%)
- Power Consumption:
- 100GB, 200GB, 400GB: 2.7 W Typ.
- 800GB: 3.6 W Typ.
- Energy Consumption Efficiency:
- 100GB, 200GB, 400GB: 19,500 IOPS/W
- 800GB: 24,400 IOPS/W
- Dimensions (W)x(D)x(H)
- 100GB, 200GB, 400GB: 69.85 mm x 100.45 mm x 7.0 mm
- 800GB: 69.85 mm x 100.45 mm x 15.0 mm
- 100GB, 200GB, 400GB: 70 g (max)
- 800GB: 170g (max)
- Temp - Operating: 0° to 55°C
- Temp - Non-Operating: -40° to 70°C
- Vibration - Operating: 21.27 m/s²
- Vibration - Non-Operating: 159.74 m/s²
- Shock - Operating: 9,800 m/s² (1,000G 0.5ms, ½ sine)
- Shock - Non-Operating: 9,800 m/s² (1,000G 0.5ms, ½ sine)
- Power Loss Protection
- Product Life: 5 years or the max total bytes written (TBW) per model capacity
- 100GB: 5.475PB TBW
- 200GB: 10.95PB TBW
- 400GB: 21.9PB TBW
- 800GB: 43.8PB TBW
Design and Build
Many of Toshiba’s new enterprise SSDs use a 7mm z-height rather than 15mm, which remains the dominant form factor among its competitors. The PX02SS uses a 2.5-inch, 7mm stamped aluminum body for every capacity except for the 800GB model, which uses a 15mm chassis.
The PX02SS uses a dual-port 12Gb/s SAS3 interface which is generally backwards-compatible with 6Gb/s SAS2 backplanes, although in the case of the PX02SS, Toshiba varys slightly from the SAS2 spec. In our case we found that our Lenovo ThinkServer, which is compliant with the SAS2 protocol in regards to the Power Disable function, would not power on the PX02SS SSD. The PX02SS SSD was designed for SAS3 systems, and other non fully-compliant SAS2 servers that leave the the Power Disable connection open. The remedy for using the PX02SS in scenarios with fully compliant SAS2 hardware is chosing a special version of the SSD that can work around that difference, or by physically cutting and removing a resistor from the SSD manually to make it not detect that Power Disable circuit.
Inside, there is a Marvell co-branded TC58NC9036GTC SAS 12Gb/s compatible controller, the same as the PX02SM. Our 400GB review model also has 16 24nm eMLC Toshiba NAND die packages.
Testing Background and Comparables
The PX02SS uses a Marvell TC58NC9036GTC controller and 24nm Toshiba eMLC NAND with an interface that supports SAS 12Gb/s. The StorageReview Enterprise Test Lab uses a Supermicro SuperStorage Server 2027R-AR24NV as our SAS3 testbed, featuring:
- 2 x Intel Xeon E5-2687 v2 (3.4GHz, 25MB Cache, 2-cores)
- Intel C602 Chipset
- Memory - 256GB (16 x 16GB) 1333Mhz Micron DDR3 Registered RDIMMs
- Windows Server 2012 Standard - 100GB Micron RealSSD P400e Boot SSD
- 3 x Supermicro SAS3 HBAs (LSI SAS 3008 Controllers)
- 100GB Micron P400e Linux CentOS 6.3 boot
- 200GB Micron P400m Windows Server 2012 boot
- 100GB Micron P400e Linux CentOS 6.3 boot (Sysbench) w/ Micron M500 960GB for Database storage
- Mellanox ConnectX-3 Dual-Port VPI PCIe 3.0 Adapter
To get a sense of how the PX02SS stacks up against other eMLC SSDs that currently available with a 12Gb/s SAS interface, we will chart the results of our PX02SS tests next to those from 400GB and 800GB editions of the Toshiba PX02SM and HGST's SSD800MM:
- Hitachi SSD800MH (400GB, Intel co-branded DB29AA11B0 controller, Intel 25nm MLC NAND, 12.0Gb/s SAS)
- Toshiba PX02SM (400GB, Marvell co-branded TC58NC9036GTC controller, Toshiba 24nm eMLC NAND, 12Gb/s SAS)
- Toshiba PX02SM (800GB, Marvell co-branded TC58NC9036GTC controller, Toshiba 24nm eMLC NAND, 12Gb/s SAS)
Application Performance Analysis
In order to understand the performance characteristics of enterprise storage devices, it is essential to model the infrastructure and the application workloads found in live production environments. Our first three benchmarks of the Toshiba PX02SS are therefore the MarkLogic NoSQL Database Storage Benchmark, MySQL OLTP performance via SysBench and Microsoft SQL Server OLTP performance with a simulated TCP-C workload.
Our MarkLogic NoSQL Database environment requires groups of four SSDs with a usable capacity of at least 200GB, since the NoSQL database requires roughly 650GB of space for its four database nodes. Our protocol uses an SCST host and presents each SSD in JBOD, with one allocated per database node. The test repeats itself over 24 intervals, requiring between 30-36 hours total for the SSDs in this class. MarkLogic records total average latency as well as interval latency for each SSD.
The Toshiba PX02SS fared well in our MarkLogic NoSQL database benchmark, coming in behind the Hitachi SSD800MM, but with a solid lead ahead of its brother the PX02SM.
The HGST SSD800MM maintained a decisive lead over the comparables, with only occasional spikes during merge read and merge write operations.
The PX02SS improved substantially over what we previously recorded on the PX02SM SAS3 SSD. Much of its latency remained below 10ms over the duration of the test.
The PX02SM architecture is not optimized for the access patterns of our NoSQL workload, triggering latencies that spiked at or above the normalized maximum value of 9ms throughout the protocol.
The next application benchmark consists of a Percona MySQL OLTP database measured via SysBench. In this configuration, we use a group of Lenovo ThinkServer RD630s as database clients and the database environment stored on a single drive. This test measures average TPS (Transactions Per Second), average latency, as well as average 99th percentile latency over a range of 2 to 32 threads. Percona and MariaDB are using the Fusion-io flash-aware application APIs in the most recent releases of their databases, although for the purposes of this comparison we test each device in their "legacy" block-storage modes.
The Toshiba PX02SS distinguished itself as the top overall performer in our MySQL benchmark once the workload increased beyond four threads. The HGST SSD800MM remained competitive with the PX02SS throughout its second-place performance.
Above three threads, the PX02SS also slightly outperforms the HGST SSD800MM in terms of average latency during the MySQL OLTP benchmark.
When faced with worst-case latency during the MySQL benchmark, the Toshiba PX02SS had a second-place performance which closely matched the 99th percentile latencies of the Toshiba PX02SM 800GB.
StorageReview’s Microsoft SQL Server OLTP testing protocol employs the current draft of the Transaction Processing Performance Council’s Benchmark C (TPC-C), an online transaction processing benchmark that simulates the activities found in complex application environments. The TPC-C benchmark comes closer than synthetic performance benchmarks to gauging the performance strengths and bottlenecks of storage infrastructure in database environments. Our SQL Server protocol uses a 685GB (3,000 scale) SQL Server database and measures the transactional performance and latency with a 30,000 VU Load.
For this review, we compare the three SSD platforms, with each configured as a mirrored storage space by Windows Server. The HGST took the lead with 6,315.4 transactions per second for 30k users, with both Toshiba comparables within 100TPS of the SSD800MM's performance.
The latency results during the Microsoft SQL TPC-C workload reveals more significant differences between the two manufacturers, as the HGST SSD800MM maintains a commanding latency advantage over the PX02SM and PX02SS.
Enterprise Synthetic Workload Analysis
Flash performance varies as the drive becomes conditioned to its workload, meaning that flash storage must be preconditioned before each of the fio synthetic benchmarks in order to ensure that the benchmarks are accurate. Each of the comparable drives are secure erased using the vendor's tools and preconditioned into steady-state with a heavy load of 16 threads and an outstanding queue of 16 per thread.
Preconditioning and Primary Steady-State Tests:
- Throughput (Read+Write IOPS Aggregate)
- Average Latency (Read+Write Latency Averaged Together)
- Max Latency (Peak Read or Write Latency)
- Latency Standard Deviation (Read+Write Standard Deviation Averaged Together)
Once preconditioning is complete, each device is then tested in intervals across multiple thread/queue depth profiles to show performance under light and heavy usage. Our synthetic workload analysis for the Toshiba PX02SS uses two profiles which are widely used in manufacturer specifications and benchmarks.
- 100% Read and 100% Write
- 70% Read/30% Write
At no point during preconditioning for the 4k synthetic benchmark did the PX02SS equal the throughput performance of the HGST SSD800MM, but it did sustain burst performance longer than its PX02SM peers and reached steady state with notably higher performance than either of the PX02SMs.
Average latency results during 4k preconditioning also suggest that the PX02SS is able to offer tighter latencies than the PX02SM for write operations. The PX02SS provides a burst of low-latency performance that lasts almost 90 minutes and then quickly settles into a consistent average latency performance below 6ms.
The PX02SS continues to deliver a strong second place performance to the HGST SSD800MM in our review of maximum latency values from the 4k preconditioning operation.
Standard deviation results make it easier to visualize how consistent the measured latencies were throughout the preconditioning process. While not as steady as the HGST comparable, the Toshiba PX02SS has the most consistent preconditioning latencies among the other 12Gb/s eMLC SAS SSDs we have benchmarked to date.
With preconditioning for the 4k benchmarks complete, the PX02SS reached 117,204 read IOPS and 42,507 write IOPS placing it second to the HGST SSD800MM.
The PX02SS experienced an average latency of 2.18ms for read operations and 6.02ms for write operations in the 4k benchmark.
The PX02SS had the best maximum read latency values among the comparables at 13.3ms. The SSD800MM came out ahead for maximum write latency wth 39.0ms.
The PX02SS maintained more consistent latency results than either of the other two Toshiba SSDs we have evaluated on the SAS3 testbed to date.
Our next workload uses 8k transfers with a ratio of 70% read operations and 30% write operations. After a burst period which saw the HGST SSD800MM and the Toshiba PX02SS vying for best throughput performance, the drives reached steady state with the PX02SS in a close second to the SSD800MM near 62,500 IOPS.
Average latency results from the 8k 70/30 preconditioning reflect the same early competitiveness between the PX02SS and the SSD800MM during burst followed by the PX02SS maintaining a consistent second-place performance.
In terms of the maximum measured latencies during 8k 70/30 preconditioning, the Toshiba PX02SS was much more like the PX02SM comparables than HGST's SSD800MM as the drives approached steady state.
In terms of standard deviation for the averaged latencies, the PX02SS spends the majority of the preconditioning period second to the HGST SSD800MM.
Once the drives are preconditioned, the 8k 70/30 throughput benchmark varies workload intensity from 2 threads and 2 queue up to 16 threads and 16 queue. Measuring 8k throughput, the PX02SS outpaces the SSD800MM on many workloads.
Measuring average latency across a range of thread counts and queue depths with 8k 70/30 operations, the PX02SS performs much more like the HGST comparable than the other Toshiba eMLC SSDs.
Maximum latency values during the 8k 70/30 benchmark were less decisive, although the HGST SSD800MM began to more clearly outperform the other comparables at the maximum workload.
Standard deviation calculations indicate that the Toshiba PX02SS experienced more consistent latency results during the 8k 70/30 benchmark than the PX02SM but was not able to deliver as narrow a range of variation as the HGST SSD800MM.
Toshiba is broadening its role in the SSD market at the same time that enterprise SSDs are beginning the migration from 6Gb/s SAS2 to 12Gb/s SAS3. The PX series of eMLC SSDs provides Toshiba with an opportunity to introduce an expanded enterprise SSD lineup that can demonstrate its in-house capacity to engineer its widely-used MLC NAND into Toshiba-branded drives that take advantage of the increased bandwidth of the SAS3 standard. The breadth of their portfolio also means they have a solution for almost all of the popular SSD use cases.
HGST's SSD800MM remains the highest-performing SAS3 SSD we have benchmarked in most tests, but the PX02SS takes a decisive second place in our analysis of this emerging market. Giving up half of the capacities of the PX02SM, the Toshiba PX02SS gains notable performance improvements across the board over its PX series sibling. The PX02SS is also able to offer high write endurance which will also appeal to customers looking for a high-performance SSDs for intensive environments such as production OLTP databases. The PX02SS in fact performed very well in our Sysbench MySQL OLTP benchmark, pulling ahead of the HGST SSD800MM. We also saw strong low-queue depth performance from the PX02SS in our 8k 70/30 mixed workload, which put it in the lead up until the higher thread/queue levels. The only letdown came in SQL testing where the drive did well in terms of TPS but posted latencies that were higher than expected.
- Long endurance in terms of lifetime total bytes written
- Consistently good performance throughout the benchmarks
- 7mm form factor gives it more flexibility in deployment
- Poor latency results in the SQL Server OLTP benchmark
- Limited to 800GB top end capacity
- Not fully SAS2 compliant in regards to Power Disable feature
The Bottom Line
The PX02SS puts Toshiba's eMLC NAND and PX-series architecture to work, in a performance-oriented SSD that demonstrates the utility of the 12Gb/s SAS3 standard while also delivering high-endurance for write-heavy applications.