Sequential Transfer Rate vs. Random Access Time
Since the inception of StorageReview.com, readers have been used to regarding areal density, linear data density, and thus sequential transfer rate in a very important light. Examine this poll result, for instance:
|What contributes most to hard disk speed?|
Spindle Speed - 599
Seek Time - 258
Data Density - 515
Buffer Size - 72
Total Participants: 1444
In the opinion of StorageReview.com readers, Data Density, and thus STR, takes a commanding second place next to only spindle speed as the most important overall specification in determining performance. Seek Time, on the other hand, comes in a distant third.
Through recent tests in WinBench 99, however, empirical results indicated that STR had relatively little effect upon overall drive performance. Today, it should be clear that steadily-increasing transfer rates have in effect "written themselves out" of the performance equation. Let's examine a few examples from the theoretical side:
Drive A has a 13 millisecond access time and a 20 MB/sec transfer rate. Drive B has a 13 millisecond access time and a 30 MB/sec transfer rate. In all other cases, the two units are identical. Just how much faster is drive B than A? Let's take an easy example, SR's IOMeter Database Access Pattern. In this particular case (which isn't far off from typical workstation usage), Drive A's average I/O operation will take 13 milliseconds + (8k block / 20,000k transfer rate) = 13.4 milliseconds. Drive B, with its superior transfer rate, would take 13.26 milliseconds to complete the transaction. Not nearly as significant as we initially thought, is it?
Let's take the example one step further with a look at STR in an asymptotic nature. That's right, assume drive B possesses an infinitely fast transfer rate. In this case, Drive B bests drive A by a 13ms vs. 13.4 ms margin. That's right, a 3% margin.
There are some who won't buy that the Database Access Pattern or even the Workstation Access Pattern are typical of today's usage (though a quick read over this should convince one otherwise). Fair enough, let's take a look at what it would take for STR to be at least as significant as random access time. In the case of Drive B, with a 13 millisecond access time and 30 MB/sec transfer rate (a level which today's ATA drives are just starting to reach), the typical transfer once the actuator and platter were in place would have to be 390k (0.013ms * 30,000k/sec) for STR and access time to have an equal effect on performance! Considering the swapped nature of most large files, the smaller size of many others, and fragmentation, it's clear that such a scenario is not representative of today's disk performance.
Judging from the above examples, it should be clear that random access time is vastly more important than sequential transfer rate when it comes to typical disk performance. Thus, the reordered "hierarchy" of important quantifiable specs would read:
- Seek Time
- Spindle Speed
- Buffer Size
- Data Density
Note that an important yet quantitatively-immeasurable factor is the nebulous drive electronics/firmware/algorithms package. Both of our benchmarks (WinBench 99 and IOMeter) are strongly influenced by these factors, enough so that they would easily place at least in 3rd place right behind spindle speed. This "algorithms are important" idea is one of the few places that WB99 and IOMeter agree (although not in the same fashion!). So, let's take a look at...
WinBench 99 vs. IOMeter
The often-contrary nature between StorageReview.com's new set of benchmarks begs the question: What's more important, WinBench 99 or IOMeter? In the past we've admitted how we use each drive we benchmark in our own personal systems to catch any cases where manufacturers are obviously padding WinBench results. Such a case arose with the Western Digital Caviar AC31300. Despite its lofty WinBench 98 scores, the drive proved to stumble behind with a newer measure that at the time was more impervious to irrelevant optimizations, WinBench 99. Recently in our older testbed, we've found that the Maxtor DiamondMax Plus 40 blows away the competition (including 10k SCSI drives!). Our new testbed, especially with IOMeter, reveals that the differences aren't nearly as significant, something matching our observations as we've had an opportunity to use these units over time.
In general, however, it's safe to say that we've been satisfied with the general correlation between WB99 results and perceived performance under normal, everyday use (something that would fall in between Light and Moderate Loads in our current IOMeter suite). With exhaustive new-testbed benchmarks completed, we've had the opportunity to take a break and place the testbed's 700 MHz processor into a personal machine that has been running at 400 MHz for the longest time. Aside from some very obvious performances increases when running a handful of today's state-of-the-art games, the machine as a whole doesn't really strike us as "more responsive." Perhaps this is because of the 384 megs of memory, or its two hard disks, the Atlas 10k II and Cheetah 36LP. This reminds us of numerous upgrades in the past, however, where a large upgrade didn't exhibit an immediately noticeable performance increase; upon reverting to the old component for control tests, however, the immense difference was much more easily perceived. We're willing to bet that the same case will apply here once the 700 MHz P3 must return to the testbed to benchmark a new drive.
To us, the moral of this story is that although we've used every drive in our own systems, the correlation between WB99 and personal use is admittedly limited to detection of obvious outliers. Further compounding the ambiguity was the "progressive" nature in which these drives entered our system. Roughly speaking, each new drive that we tried out was a bit "better" than the ones we used a few months ago.
As a result, in an attempt to create a much more stark contrast, we underwent rapid-fire, repeated switching between two dramatically different (in terms of WB99) drives: the Barracuda 9LP and the DiamondMax Plus 40. In this extended trial, with the switch going both ways, it became clear that the state-of-the-art ATA drive was not perceptively faster. Indeed, in some cases, we felt the Barracuda possessed the edge.
Another benefit in IOMeter's nature is its rather low-level nature. The fact that it doesn't possess a scripted access pattern to be played back (rather, it creates its own loads) makes it much more difficult for manufacturers to "tune" a drive for better scores without actually resulting in a proportional increase in actual usage. Thus, we have high expectations for IOMeter- we hope the benchmark will be able to carry us for some time.
The net result is that StorageReview.com from this point forwards will weigh IOMeter results, particularly of the Workstation Access Pattern, more highly than WinBench 99. This isn't to say that WinBench 99 is useless... in particular, its low-level measures such as the Disk/Read Transfer Time and Disk Access Time are still unrivaled. But over the last month and a half, overall, we've become more and more convinced that it is IOMeter, not WinBench, that correlates best to "real world" disk usage. Even so, we'll continue to use WinBench 99 until ZD deploys WinBench 2001.
Now then, let's take a look at what ramifications these findings have on the current StorageReview.com Leaderboard:
5400rpm ATA Drives
This unassuming category is where the largest quandary exists. The Western Digital Caviar WD307AA is the decisive leader in WinBench 99. The same drive, however, brings up the rear of our new IOMeter database. On the other hand, the Quantum Fireball lct series, though sluggish in WinBench 99, dominate in IOMeter thanks to their speedy access times. The edge goes to the lct10 which, despite slightly lower IOMeter scores compared to the lct(8), sports a higher sequential transfer rate, useful in applications where STR may be important.
7200rpm ATA Drives
Here we witness an amazing struggle. WD's Expert tops the collection of all tested ATA drives in IOMeter. The Expert, alas, is no longer a current product however, being phased out in favor of the 7200rpm Caviar... a unit that doesn't perform nearly as well in Intel's measure. Maxtor's DiamondMax Plus 6800 and DiamondMax Plus 40 drives finish within a few percentage points of each other in IOMeter. Interestingly, the older-generation Plus 6800 comes out on top (by negligible margins) in most cases. The Plus 40, however, enjoys a decisive advantage in transfer rates (and WinBench 99 scores) and thus narrowly retains the 7200rpm ATA Leaderboard spot for the same reasons the lct10 trumps its smaller brother in the 5400rpm category.
7200rpm SCSI Drives
Our new database is admittedly very thin when it comes to 7200rpm SCSI drives, containing only an older Seagate unit and the state-of-the-art Quantum Atlas V. Obviously, the Atlas V is the fastest 7200rpm SCSI unit we've tested in the new system. A review of Seagate's new Barracuda 18XL is forthcoming, however. This may change Quantum's comfortable hold in our SCSI categories.
10,000rpm SCSI Drives
When all is said and done, the Quantum Atlas 10k II still retains its title of "world's fastest drive." Interesting to note, however, are the competitive IOMeter results that the Seagate Cheetah 36LP turns in despite it's obviously slower physical design. Had the Cheetah featured an actuator capable of 4.7ms seeks, the outcome would have likely been different.
This concludes StorageReview.com's 2nd anniversary rollout of a new testbed. Over the next several weeks, we're going to run the old and new systems side-by-side to further ease the transition. We'd like to thank everyone for joining us during this very interesting week. The blitz, however, is not quite finished yet. We've got a new SR team member waiting in the wings, ready to cover a topic that's been hotly demanded by readers. And as mentioned above, Seagate's promising Barracuda 18XL has a review coming right up. Stay tuned!