December 21st, 2015 by StorageReview Enterprise Lab
Intel SSD DC P3608 AIC NVMe SSD Review
The Intel DC P3608 is a high-performance enterprise NVMe SSD that leverages the add-in-card (AIC) form factor aimed at the database, HPC and real-time analytics. The half-height, half-length (HHHL) cards offer capacities up to 4TB and performance at the top end of 5GB/s sequential read and 3GB/s sequential write, along with 850,000 random 4K read IOPS. The P3608 extends the P3600 series which comes in both 2.5" and AIC form factors. The P3608 effectively combines two P3600s onto one AIC, which yields a doubling of capacity and performance from a single P3600 drive.
The P3608 sits in middle-ground territory within the Intel enterprise SSD portfolio. Where the P3500 family is designed for read-intensive workloads and the P3700 family for write-intensive workloads, the P3600 and P3608 find themselves distinctly targeted to what amounts to the bulk of enterprise use cases, mixed-workloads. As such the P3608 is Intel's multitool, able to handle a variety of applications hitting it at once, a pretty typical scenario in most enterprise environments. The fact that it's an AIC (PCIe 3.0 x8) makes deployment easy too, any modern server will handle the cards without issue, which is not the case for the 2.5" form factor NVMe drives that require a specialized server backplane. We even took the P3608 and used it for testing Dell's latest launch of single-processor servers and found the combination to be quite impressive.
The P3608 is a vertically integrated solution, the NAND, controllers and firmware are all from Intel. The drive uses 20nm MLC NAND with what Intel calls "high endurance technology." This leads to endurance spec of 3 drive writes per day over five years. This translates into roughly 22PB of writes for the 4TB P3608.
The Intel P3608 ships in 1.6TB, 3.2TB and 4TB capacities; our review is of the 1.6TB capacity.
Intel DC P3608 Specifications
- Capacities: 1.6TB, 3.2TB, 4TB
- NAND: 20nm HET MLC NAND
- 4K Random Read IOPS: 850,000
- 4K Random Write IOPS: 150,000 (1.6TB), 80,000 (3.2TB), 50,000 (4TB)
- Seq. Read Write MB/s: 5,000/2,000 (1.6TB), 4,500/2,600 (3.2TB), 5,000/3,000 (4TB)
- Endurance (PBW):
- 8.76 (1.6TB)
- 17.52 (3.2TB)
- 21.90 (4TB)
- Power (Read/Write):
- 18/30W (1.6TB)
- 18/35W (3.2TB)
- 20/40W (4TB)
- Power Idle: 11.5W
- Enhanced Power Loss Management
- End-to-End Data Protection
- Dual NVMe Controller
- Warranty: 5-year
Design and Build
The Intel SSD DC P3608 is certainly a beast of a drive with its half-height, half-length PCIe x4 form factor. A generously sized heat sink takes up most of the card, as the P3608 does not use any fan-based cooling. The card itself is very well designed with the aluminum cover with a panel on top that consists of the company’s branding.
The x8 PCIe 3.0 interface is located alongside the bottom of the card.
Removing the heat sink shows the NAND packages, each of which leverages Intel’s 20nm NAND technology. The other side of the PCB shows Intel’s new controller.
Testing Background and Comparables
The StorageReview Enterprise Test Lab provides a flexible architecture for conducting benchmarks of enterprise storage devices in an environment comparable to what administrators encounter in real deployments. The Enterprise Test Lab incorporates a variety of servers, networking, power conditioning, and other network infrastructure that allows our staff to establish real-world conditions to accurately gauge performance during our reviews.
We incorporate these details about the lab environment and protocols into reviews so that IT professionals and those responsible for storage acquisition can understand the conditions under which we have achieved the following results. None of our reviews are paid for or overseen by the manufacturer of equipment we are testing. Additional details about the StorageReview Enterprise Test Lab and an overview of its networking capabilities are available on those respective pages.
We tested the Intel SSD DC P3608 inside the AIC NVMe SSDs:
- Memblaze PBlaze4 3.2TB
- Huawei HHHL ES3000v2 1.6TB
- SanDisk SHHL PX600 2.6TB
- Huawei HHHL ES3000v2 3.2TB
- HGST HHHL SN100 3.2 TB
Application Workload Analysis
The P3608 didn't have enough capacity to run 4x SQL VMs, so it was excluded from SQL Server benchmarking. If Intel sends a larger card in the future we will benchmark it for SQL and ammend this section of the review.
With that said, our application benchmark consists of a Percona MySQL OLTP database measured via SysBench. This test measures average TPS (Transactions Per Second), average latency, as well as average 99th percentile latency. Percona and MariaDB are using the Fusion-io flash-aware application APIs in the most recent releases of their databases, although for the purposes of this comparison we test each device in their "legacy" block-storage modes.
In the average transactions per second benchmark, the Intel P3608 was slightly edged out by the Memblaze drive in aggregate performance, with 5,877.9 TPS and 5,954.8 TPS, respectively.
When looking at average latency results, the Intel drive boasted the best top performance, with individual VMs running between 20.31ms through 23.56ms and aggregate latency of 21.89ms. The Memblaze, however, had the best aggregate with 21.50ms.
In terms of our worst-case MySQL latency scenario (99th percentile latency), the Intel showed the best top performance, with VMs running between 41.73ms and 51.89ms while the top aggregate was recorded by the SanDisk drive, which had 41.92ms.
Enterprise Synthetic Workload Analysis
Flash performance varies as the drive becomes conditioned to its workload, meaning that flash storage must be preconditioned before each of the fio synthetic benchmarks in order to ensure that the benchmarks are accurate. Each of the comparable drives are secure erased using the vendor's tools and preconditioned into steady-state with a heavy load of 16 threads and an outstanding queue of 16 per thread.
- Preconditioning and Primary Steady-State Tests:
- Throughput (Read+Write IOPS Aggregate)
- Average Latency (Read+Write Latency Averaged Together)
- Max Latency (Peak Read or Write Latency)
- Latency Standard Deviation (Read+Write Standard Deviation Averaged Together)
Once preconditioning is complete, each device is then tested in intervals across multiple thread/queue depth profiles to show performance under light and heavy usage. Our synthetic workload analysis for the Intel P3608uses two profiles, which are widely used in manufacturer specifications and benchmarks. It is important to take into consideration that synthetic workloads will never 100% represent the activity seen in production workloads, and in some ways inaccurately portray a drive in scenarios that wouldn't occur in the real world.
- 100% Read and 100% Write
- 70% Read/30% Write
In our throughput 4k write preconditioning test, the Intel started at roughly 480,000 IOPS and hitting a steady-state at around 162,000 IOPS mark. The top overall performing drive here was the Huawei 3.2TB drive.
Next we look at average latency. Here, the Intel drive boasted initial latency of roughly 0.5ms, taking a spike around the 20 minute mark. It finished in the middle of the pack when it reached its steady-state at just over 1.6ms. The least consistent drive was the HGST.
When measuring max latency, the Intel showed no major spikes during the entirety of our test and was certainly the most stable drive. Overall, the Memblaze drive had the least consistent results with major spikes throughout the benchmark.
Standard deviation calculations is designed make it easier visualize the consistency of the SSD latency performance results. In this scenario, the Intel drive had the most stable results, hovering around the 1.5ms during most of the test.
During the primary 4k synthetic benchmark, the Intel drive was by the top performer in the read column with an impressive 851,693 IOPS (reaching 157,940 IOPS write) while the Huawei 3.2TB drive showed the best write performance with 229,914 IOPS.
Looking at average latency showed the Intel drive with the best overall read performance again measuring just 0.30ms read (1.62ms write). The Huawei 3.2TB drive posted the best write latency again, with 1.11ms.
In max latency, the Intel drive moved to the back of the pack in read latency with 9.1ms; however, it had the best write results with just 19.1ms. The top performer in reads was the Memblaze drive though it had significantly higher write latency.
Looking at standard deviation shows the Intel move towards the middle of the pack in read latency with 0.218ms and an average write latency of 1.667ms. The top performer here was the Memblaze drive, which boasted 0.107ms in reads.
Our next workload uses 8k transfers with a ratio of 70% read operations and 30% write operations. Again, we will start off with the preconditioning results before switching to the main tests. In throughput, the Intel drive showed incredible burst speeds reaching over 700,000 IOPS; however, performance dipped significantly (and quickly) ending up with a steady-state of roughly 180,000 IOPS for placing at the bottom of the pack.
Average latency told a similar story, with the Intel drive showing a leading initial latency of just 0.4ms and quickly increasing to a steady-state over 1.5ms placing it at the bottom of the leaderboard with the Huawei 1.6TB drive. The top performing drive here was the Huawei 3.2TB with the Memblaze being the most stable by a significant margin.
When looking at max latency, the Intel drive was easily the most consistent of the bunch, showing several spikes just over 16ms throughout. The most unstable drive was the Memblaze, which was similar to its max readings during our 4K test.
Standard deviation showed pretty inconsistent results across the board during the first part of the test. The Intel drive started at roughly 0.5ms while reaching a steady state at around 1.4m. The top performing SSD was by far the Huawei 3.2TB card, which showed great consistency ending up at just over 0.9ms
After we fully preconditioned the Intel P3700 drive, we put it through our main 8k 70/30 test. In throughput, the Intel drive showed middle of the pack performance with identical performance to three of the other drives throughout.
Average latency mirrored the throughput results, with the Intel, Memblaze, HGST, and 1.6TB performing neck and neck to until the very end. The top performing drive here was the 3.2TB Huawei drive, which ended up just under 0.9ms.
Looking at max latency showed the Intel drive with very consistent results, reaching at just under 17ms in the terminal for top spot among the tested SSDs. The Huawei had best overall results, as it only surpassed the Intel drive in latency at the very end.
Standard deviation demonstrated very similar performance between all of the drives until the very end of the test; however, the Huawei 3.2TB drive pulled away around the 16T2Q mark, posting the best overall results at just over 0.9ms.
Available in capacities up to 4TB, the Intel P3608 leverages the AIC form factor and is designed for database, HPC and real-time analytics applications. Though it sits somewhere in the middle of the performance spectrum within their family of enterprise drive, Intel has certainly showed us that the NVMe SSD can contend with the other power-house drives out there. The P3608 works best with mixed workloads, with the ability to flourish when a variety of applications hit it all at once; this is pretty common-place in most enterprise environments. As we mentioned before, the Intel drive is easy to deploy due to its AIC (PCIe 3.0 x8) form factor, as pretty much any modern server can easily handle that type of card.
Like the P3700 family, the P3608 drives are vertically integrated solutions, as the Intel produces all of the components including the controller, NAND (20nm MLC) and firmware. This efficient design type allows Intel offer much better support, streamlined enhancements, and firmware updates, as they understand the drive's characteristics.
When looking at performance, the Intel DC P3608 showed some great results throughout our tested benchmarks, leading in quite a few of our tested workloads. That said, we pitted the Intel drive against the Huawei ES3000 (3.2TB and 1.6TB), Memblaze PBlaze4 3.2TB and the HGST SN100 NVMe HHHL SSDs. In our Sysbench tests, we saw some overall impressive performance for the Intel DC P3608. In the average transactions per second benchmark, the Intel P3608 boasted an aggregate of 5,877.9 TPS, which was just slightly less than the Memblaze drive. When looking at average latency results, the Intel drive boasted the best top performance in individual VMs, running between 20.31ms through 23.56ms and aggregate latency of 21.89ms. In terms of our worst-case MySQL latency scenario, the Intel showed the best top performance (again in individual VMs), which ran between 41.73ms and 51.89ms.
During our main Synthetic benchmarks, the Intel DC P3608 posted a very impressive leading 4k throughput of 851,693 IOPS read while writes hit 197,940 IOPS. Average latency also showed top read results with 0.30ms with writes of 1.62ms. In our 8k transfers (which consists of a ratio of 70% read operations and 30% write operations), the Intel drive slowed down a bit in throughout and average latency, with results that placed it in the middle of the pack though its max latency and standard deviation readings were the top among the leaderboard.
- Great overall performance
- Integrated solution
- Up to 4TB in capacity
- Capacity split across two storage pools
The Intel P3608 is an AIC SSD that is aimed at database, HPC, and real-time analytics and brings a strong performance punch with it.