December 5th, 2014 by StorageReview Enterprise Lab
Toshiba HK3R2 Review
The Toshiba HK3R2 series is the company's new line of high-performance enterprise SSDs and direct successor to the HK3R eSSD family. Specifically designed for read intensive workloads, the HK3R2 has a maximum capacity of 960GB in a 2.5" form factor, supports a SATA 6Gb/s interface and leverages Toshiba's second-generation A19nm MLC NAND. The HK3R2 also features vital enterprise functionality, including full power loss and to-end protection as well as proprietary QSBC (Quadruple Swing-By Code) error-correction technology, the latter which helps to protect and secure data from corruption caused by NAND flash memory media wear. Its counterpart, the HK3E2, was also announced by Toshiba this week, though it's demographic targets businesses looking for value-oriented intensive workload SSDs instead.
The read intensive applications and workloads the HK3R2 was designed for include read caching, video streaming, and data center storage. As such, Toshiba has quoted pretty impressive read speeds for all capacities with 500MB/s as well as good random 4K read performance (also for all capacities) at 75,000 IOPS. That being said, write speed and performance varies by capacity, with sequential write speeds being stated as 120MB/s for the 120GB, 270MB/s for the 240GB, and 400MB/s for both the 480GB and 960GB. The random performance are quoted to boast 4,000 IOPS for the 120GB, 10,000 IOPS for the 240GB, 12,000 IOPS for the 480GB, and 14,000 IOPS for the 960GB. Toshiba has also built the HK3R2 to be pretty durable for the targeted use case, supporting one drive write per day under random 4K workloads.
The Toshiba HK3R2 series comes in 120GB, 240GB, 480GB, and 960GB capacities with a 5-year warranty. Our review is for the 4x960GB model.
- Drive Capacities:
- NAND Technology: Toshiba A19nm MLC
- Drive Interface: SATA 3.2 (6.0 Gbit/s, 3.0 Gbit/s, 1.5 Gbit/s)
- Logical Blocks (LBA): 512B
- 4KiB Random Read (Sustained): 75,000 IOPS
- 4KiB Random Write (Sustained):
- 4,000 IOPS (120GB)
- 10,000 IOPS (240GB)
- 12,000 IOPS (480GB)
- 14,000 IOPS (960GB)
- 64KiB Sequential Read (Sustained): 500 MiB/s
- 64KiB Sequential Write (Sustained):
- 120 MiB/s (120GB)
- 270 MiB/s (240GB)
- 400 MiB/s (480GB)
- 400 MiB/s (960GB)
- Product Life: 5 years
- Power Requirements
- Voltage: 5V (+/- 5%)
- Power Consumption: 4.5W Typical
- Energy Consumption Efficiency 16,600 IOPS/W
- Dimensions (W)x(D)x(H): 69.85 mm x 100.45 mm x 7.0 mm
- Weight: 60 grams
- Availability (hrs/day x days/wk): 24 x 7
- Endurance - Total Bytes Written:
- 220TB (120GB)
- 440TB (240GB)
- 880TB (480GB)
- 1,760TB (960GB)
- Warranty: 5-years
Design and build
The Toshiba HK3R2 uses a 7mm z-height form factor stamped aluminum body for every capacity. As is the case with most enterprise drives, there are no frills when it comes to the design. It simply has a product sticker on the drive with a barcode, drive specifics, and other information including certifications and designations. The HK3R2 uses a SATA 6.0Gb/s interface.
The side profiles show the four screw holes that enable the Toshiba HK3R2 to be mounted with ease.
Removing the cover is fairly easy, just remove the 4 screws, and pry the top open. Our 960GB review model uses a Toshiba-branded controller while the storage comes from Toshiba's A19nm MLC NAND die packages.
Testing Background and Comparables
The StorageReview Enterprise Test Lab provides a flexible architecture for conducting benchmarks of enterprise storage devices in an environment comparable to what administrators encounter in real deployments. The Enterprise Test Lab incorporates a variety of servers, networking, power conditioning, and other network infrastructure that allows our staff to establish real-world conditions to accurately gauge performance during our reviews.
We incorporate these details about the lab environment and protocols into reviews so that IT professionals and those responsible for storage acquisition can understand the conditions under which we have achieved the following results. None of our reviews are paid for or overseen by the manufacturer of equipment we are testing. Additional details about the StorageReview Enterprise Test Lab and an overview of its networking capabilities are available on those respective pages.
SAS and SATA enterprise SSDs are benchmarked on our second-generation enterprise testing platform based on a Lenovo ThinkServer RD630. This testing platform includes the latest interconnect hardware such as the LSI 9207-8i HBA as well as I/O scheduling optimizations geared towards best-case flash performance. For synthetic benchmarks, we utilize FIO version 2.0.10 for Linux and version 184.108.40.206 for Windows.
- 2 x Intel Xeon E5-2620 (2.0GHz, 15MB Cache, 6-cores)
- Intel C602 Chipset
- Memory - 16GB (2 x 8GB) 1333Mhz DDR3 Registered RDIMMs
- Windows Server 2008 R2 SP1 64-bit, Windows Server 2012 Standard, CentOS 6.3 64-Bit
- 100GB Micron RealSSD P400e Boot SSD
- LSI 9211-4i SAS/SATA 6.0Gb/s HBA (For boot SSDs)
- LSI 9207-8i SAS/SATA 6.0Gb/s HBA (For benchmarking SSDs or HDDs)
- Mellanox ConnectX-3 10GbE PCIe 3.0 Adapter
- Mellanox ConnectX-3 InfiniBand PCIe 3.0 Adapter
Application Workload Analysis
In order to understand the performance characteristics of enterprise storage devices, it is essential to model the infrastructure and the application workloads found in live production environments. Our first three benchmarks of the Toshiba HK3R2 are therefore the MarkLogic NoSQL Database Storage Benchmark, MySQL OLTP performance via SysBench and Microsoft SQL Server OLTP performance with a simulated TCP-C workload.
Our MarkLogic NoSQL Database environment requires groups of four SSDs with a usable capacity of at least 200GB, since the NoSQL database requires roughly 650GB of space for its four database nodes. Our protocol uses an SCST host and presents each SSD in JBOD, with one allocated per database node. The test repeats itself over 24 intervals, requiring between 30-36 hours total. MarkLogic records total average latency as well as interval latency for each SSD.
In our overall average latency tests using our MarkLogic NoSQL database benchmark, the HK3R2 performed extremely well, taking top spot in our SATA SSD group with 2.122ms.
The latency results for the majority of operations during the NoSQL benchmark below 12ms; however, the HK3R2 showed a few spikes near the end of our tests. This is not unusual SSDs, so we weren't surprised.
The Toshiba HK3R2 remained at the top of the leaderboard (from start to finish) in our MySQL benchmark with roughly 1,650 transactions per second at 32 threads workload.
The next application benchmark consists of a Percona MySQL OLTP database measured via SysBench. In this configuration, we use a group of Lenovo ThinkServer RD630s as database clients and the database environment stored on a single drive. This test measures average TPS (Transactions Per Second), average latency, as well as average 99th percentile latency over a range of 2 to 32 threads. Percona and MariaDB are using the Fusion-io flash-aware application APIs in the most recent releases of their databases, although for the purposes of this comparison we test each device in their "legacy" block-storage modes.
Again, the HK3R2 boasted great performance with the lowest average latency, even beating out the SM853T.
In our worst-case MySQL latency scenario, the Toshiba HK3R2 posted results towards the middle of the pack. The top performer here was the Samsung PM853T.
StorageReview’s Microsoft SQL Server OLTP testing protocol employs the current draft of the Transaction Processing Performance Council’s Benchmark C (TPC-C), an online transaction processing benchmark that simulates the activities found in complex application environments. The TPC-C benchmark comes closer than synthetic performance benchmarks to gauging the performance strengths and bottlenecks of storage infrastructure in database environments. Our SQL Server protocol uses a 685GB (3,000 scale) SQL Server database and measures the transactional performance and latency with a 30,000 VU Load.
The HK3R2's SQL Server Output TPS boasted top performance among the drives, though most of the comparables all had similar results.
When looking at our average latency for SQL, the HK3R2 beat out all of the tested drives (with the exception of the Samsung 845DC Pro, in which it tied), with an overall average latency of only 12ms.
Synthetic Workload Analysis
Our synthetic benchmark protocols each begin by preconditioning the target storage into steady-state with the same workload that will be used to test the device. The preconditioning process uses a heavy load of 16 threads with an outstanding queue of 16 per thread.
- Preconditioning and Primary Steady-State Tests:
- Throughput (Read+Write IOPS Aggregate)
- Average Latency (Read+Write Latency Averaged Together)
- Max Latency (Peak Read or Write Latency)
- Latency Standard Deviation (Read+Write Standard Deviation Averaged Together)
Once preconditioning is complete, each device being compared is then tested across multiple thread/queue depth profiles to show performance under light and heavy usage. Our synthetic workload analysis for the Toshiba HK3R2 uses 4k and 8k profiles which are widely used in manufacturer specifications and benchmarks.
- 4k Profile
- 100% Read and 100% Write
- 8K Profile
- 70% Read, 30% Write
- 100% 8K
During the 4K preconditioning process, the HK3R2 showed great burst results though it showed a continual drop in throughput nearing the 80 minute mark, ending just under the 24,000 IOPS mark.
The HK3R2 was third to last place on our charts in preconditioning average latency. It started off strong once again but had a steady rise in latency throughout the benchmark. That being said, it didn't show an jumps in latency and was pretty consistent.
In our maximum latency benchmark, the major showed major spikes throughout the benchmark. The Samsung 845DC Pro was by far the best drive here.
The HK3R2 came in third to last during our standard deviation part of our 4k preconditioning test, though no noticeable spikes were detected.
When looking at 4K throughput, the Toshiba HK3R2 returned to top form, as it boasted the best read throughput with 75,899 IOPS (the Samsung 845DC Pro was just behind with 74,900 IOPS). That said, the new Toshiba drive had sub par write performance (19,055 IOPS) with the 845DC Pro boasting an impressive 49,888 IOPS for top spot by a large margin.
Moving to average latency, the Toshiba HK3R2 once again had the best read performance, though it slipped quite a bit in the write category. The top performer for write latency was the Samsung 845DC Pro once again.
The HK3R2 continued its impressive performance in the read column in max latency, scoring only 20.13ms for top spot on our charts. The Samsung 845DC Pro had the best max write latency with 33.29ms.
Looking at standard deviation, the HK3R2 retained its top read performance with only 1.60ms, though it recorded sub par write performance at 8.16ms. Unsurprisingly, the Samsung 845DC Pro had the best write latency with only 2.62ms.
Our next workload uses 8k transfers with a ratio of 70% read operations and 30% write operations. Here, the HK3R2 by far had the most consistent performance up until the 180 minute mark, where it dipped a bit, ending up in fourth place just below the 31,200 IOPS mark. The top overall performer here was the Samsung 845DC Pro.
Standard deviation calculations for the 8k 70/30 preconditioning show the Toshiba HK3R2 in fourth place once again by the end of our tests where it hovered around the 8ms latency mark.
When looking at our maximum latency results for the HK3R2 during our 8k 70/30 preconditioning, the drive oscillated up to near 140ms with major spike throughout the test.
Standard deviation calculations for the 8k 70/30 preconditioning show the HK3R2 in third to last place by the end; however, it was one of the more consistent drives with only one major spike around the 180 minute mark.
Once the drives are preconditioned, the 8k 70/30 throughput benchmark varies workload intensity from 2 threads and 2 queue up to 16 threads and 16 queue. In our throughput benchmark, the HK3R2 posted middle-of-the-pack numbers peaking at around 24,000IOPS. The Samsung 845DC Pro continued its dominance with a peak IOPS of 35,000.
In our average latency test, the Toshiba HK3R2 showed performance in the group with the higher latency, placing fouth overall. Of the three drives from the group with the lowest latency, the 845DC Pro just edged out the other comparables.
In our 8k 70/30 max latency benchmark, the HK3R2 again found itself near the bottom of the pack, though no where near the last place performing drive (the OCZ Intrepid 3600). The Intel DC S3700 was the best overall drive for max latency.
Our very last test looks at standard deviation. Here, the HK3R2 was the worst performing drive by the 16T16Q mark, posting over 9ms.
The Toshiba HK3R2 series of enterprise class SSDs are designed for read intensive workloads with a maximum capacity of 960GB. They support the SATA 6.0Gb/s interface, use second-generation Toshiba A19nm MLC NAND, and come with power loss protection, end-to-end data path protection and QSBC error correction, all of which are vital data security/reliability measures for the enterprise.
Looking at application performance, the HK3R2 posted fantastic class-leading results across the board. In our MarkLogic NoSQL Database environment and SysBench tests, the new Toshiba enterprise drive was the top performer is every category with the exception of the 99th percentile test. This was headlined by a very impressive average latency in our SQL Server Output benchmark, where it was a solid 7ms better than any other drive, the previous best-in-class Samsung 845DC Pro being the only other drive to accomplish this.
When we switched to our synthetic tests, which are proving to be less useful to reflect real-world conditions, the Toshiba HK3R2 SSD continued its dominance in our 4K 100% Read/Write tests, though mainly in the read activity column (its write scores were actually quite weak compared to the other enterprise SSDs that compared it with). As for its preconditioning then 8k 70/30 benchmarks, its performance dropped and consistently ranked near the bottom of the pack in throughput and latency. As we've indicated though, the slower synthetic results don't matter much when the drive completely dominates the group in our application tests.
The net result is the Toshiba HK3R2 sets a new performance standard in the entry-enterprise SSD category, routinely dispatching its competition in our application tests and in most synthetic areas as well. The only trade-off with the drive is a modest endurance rating of one drive write per day, but that's part of the buying process. By giving up endurance, the drive can be cost effective while driving incredible throughput. This is simply a case of the enterprise flash market continuing to mature and stratify based on customer needs and usage profiles.
- Class-leading application performance in the entry-enterprise SATA group
- Drive design accounts for key enterprise reliability features
- Great 4K read performance
- Slower synthetic write performance
The Toshiba HK3R2 series of SSDs offer class-leading performance for the entry-enterprise category, complete with data protection and reliability features that ensure data integrity over the life of the drive.