Despite the precision manufacturing processes used to create hard disks, it is
virtually impossible to create a disk with tens of millions of sectors and not have some
errors show up. Imperfections in the media coating on
the platter or other problems can make a sector inoperable. A problem with a sector, if
uncorrected, would normally manifest as an error when attempting to read or write the
sector, but can appear in other ways as well. Most of us have experienced these errors on
occasion when using floppy disk drives.
Modern disks use ECC to help identify when errors occur and
in some cases correct them, however, there will still be physical flaws that ECC cannot
overcome, and that therefore prevent parts of a disk from being used. Usually these are
individual sectors that don't work, and they are appropriately enough called bad
sectors. Tracks where there are bad sectors are sometimes called bad tracks.
If you've ever used a disk information utility on a floppy disk (or on a very old hard
disk), you've likely at some point seen a report showing a few kilobytes worth of bad
sectors. However, if you run such a utility on a modern hard disk, you will normally never
see any reports of bad sectors on the disk. Why is this?
Sectors that are bad cannot be used to store data, for obvious reasons: they are bad
because they cannot be trusted to reliably write and/or reproduce the data at a later
time. It is therefore necessary for some part of the system to keep track of
where they are, and not use them. The best way for this to be done is for the drive to
detect and avoid them. If the drive does not do this, the operating system must do it. If
any bad sectors are not detected until after they have been used, data loss will probably
result.
To allow for maximum reliability then, each disk drive is thoroughly tested for any
areas that might have errors at the time it is manufactured. All the sectors that have
problems or are thought to be unreliable, are recorded in a special table. This is called defect
mapping. Some drives go even further than this, mapping out not only the sectors that
are questionable, but the ones surrounding them as well. Some drives will map out entire
tracks as a safety precaution.
On older hard disks, these problem areas were actually recorded right on the top cover
of the disk, usually in hand-writing by the technician testing the drive! This process was
necessary because low-level formatting was done by the company assembling the PC--or even
the end-user--and this information was used to tell the controller which areas of the disk
to avoid when formatting the disk. Part of the low-level format process was to have the
person doing the LLF tell the controller which sectors were bad, so it would avoid them,
and also tell any high-level format program not to try to use that part of the disk. These
markings are what cause "bad sectors" reports to show up when examining older
hard disks: these are the areas the disk has been told not to use. Since floppy disks are
low-level formatted and high-level formatted at the same time, the same situation applies,
even today. If any sectors cannot be reliably formatted, they are marked as
"bad" and the operating system will stay away from them.
While early PC users accepted that a few bad sectors on a drive was normal, there was
something distasteful about plopping down $1,000 for a new hard disk and having it report
"bad sectors" as soon as you turned it on. There is no way to produce 100%
perfect hard disks without them costing a small fortune, so hard disk manufacturers
devised an interesting compromise.
On modern hard disks, a small number of sectors are reserved as substitutes
for any bad sectors discovered in the main data storage area. During testing, any bad
sectors that are found on the disk are programmed into the controller. When the controller
receives a read or write for one of these sectors, it uses its designated substitute
instead, taken from the pool of extra reserves. This is called spare sectoring. In
fact, some drives have entire spare tracks available, if they are needed. This is all done
completely transparently to the user, and the net effect is that all of the drives of a
given model have the exact same capacity and there are no visible errors. This means that
the operating system never sees the bad areas, and therefore never reports "bad
sectors". They are still there though, just cleverly hidden.
Really, when you think about it, the hard disk companies are sacrificing a small amount
of storage for "good looks". It would be more efficient to use all of the
sectors on the disk and just map out the few bad ones. However, sometimes marketing wins
out over engineering, and it seems that more people want the warm feeling of thinking they
have a perfect drive, even if it costs them theoretical storage in the process. Today's
drives are so enormous that few people would even care much anyway about a few extra
megabytes, but that wasn't always the case!
Due to spare sectoring, a brand new disk should not have any bad sectors. It is
possible, however, for a modern IDE/ATA or SCSI hard disk to develop new bad sectors over
time. These will normally be detected either during a
routine scan of the hard disk for errors (the easy way) or when a read error is
encountered trying access a program or data file (the hard way). When this happens, it is
possible to tell the system to avoid using that bad area of the disk. Again, this can be
done two ways. At the high level, the operating system can be told to mark the area as bad
and avoid it (creating "bad sector" reports at the operating system level.).
Alternately, the disk itself can be told at a low level to remap the bad area and
use one its spares instead. This is normally done by using a zero-fill or diagnostic utility, which will
scan the entire disk surface for errors and tell the controller to map out any problem
areas.
Warning: Bad sectors on a
modern hard disk are almost always an indication of a greater problem with the disk. A new
hard disk should never have bad sectors on it; if you buy one that does have bad
sectors, immediately return it to the vendor for exchange (and don't let them tell you
"it's normal", because it isn't.) For existing hard disks, the vast majority of
time, a single bad sector that appears will soon be accompanied by friends. While you can
map out and ignore bad sectors, you should make sure to contact the vendor if you see bad
sectors appearing during scans, and make sure the data is backed up as well. Personally, I
will not use any hard disk that is developing bad sectors. The risk of data loss is too
high, and hard drives today are inexpensive compared to the cost of even an hour or two of
recovering lost data (which takes a lot more than an hour or two!).
On some disks, remapped sectors cause a performance penalty. The drive first seeks and
reads the sector header of the data, thinking it will be there; then, it sees that the
sector has been remapped, and has to do another seek for the new sector. Newer drives
using the No-ID sector format eliminate this problem by storing a format map, including
sector remaps, in the memory of the drive's controller. See
the discussion of sector format for more.
Next: Low-Level Format, Zero-Fill and Diagnostic Utilities
