Free Space and the Estimated and Actual Compression Ratios

One source of confusion in the use of compressed disk volumes relates to the amount of free space reported on the drive. This can--and will--change in sometimes unexpected ways as the compressed disk fills up, based on how compressible the files placed on the volume are. The reason is simple: the free space on a real disk volume is easily determined simply by examining how many clusters are free in the file allocation table, and multiplying by the cluster size. With a compressed volume, we don't know how much space is free until we know what will be put on the volume, because some files can be compressed a great deal, and some not at all.

In fact, when you see a report of disk space free on a compressed volume, what you are seeing is an estimate. Every compressed volume has a number associated with it called its estimated compression ratio, which is what tells the compressed volume driver how well you think the files on this volume are going to be compressed. This number can be set on a volume-by-volume basis, and should be estimated, ideally, from the actual compression ratio that the volume is using with its current files. Because it makes their tools look impressive, many compression utilities default this estimate to something like 2:1, even though the average disk volume in my experience will not compress at a figure that high. Usually 1.6 to 1.8 is more typical (depending on the settings and of course, to a great deal on what is stored on the drive).

Using the estimated compression ratio, the system will determine an estimated amount of free space by multiplying the uncompressed free space by the estimated ratio. This is what you see as free space on the drive. If you change the estimated compression ratio, the report of free disk space changes as well; in reality, you have not changed at all the capacity of the compressed volume.

As soon as you copy files to the compressed disk, they are compressed at whatever rate the compression software can manage for the type of file. A huge text file could be compressed at 3:1; a ZIP file--which is already compressed internally--will not compress at all. The amount of space these real files takes up will vary, and so the amount of free space will change, depending on what the files are.

OK, time for an example. Let's suppose we have an empty 500 MB hard disk that we want to compress. We run DriveSpace 3, say, and it sets up a 500 MB CVF on the host disk, and creates a new compressed drive called F: for example. When we look at F:, we see 1000 MB free. Why? Because the default is a 2:1 estimated compression ratio. This can be changed, of course. For now, let's leave it at 2:1. Here's what the disk looks like:

So we have 500 MB of "true" space on the host disk, in the CVF, and 1000 MB of space in the compressed disk, assuming our 2:1 ratio. Now suppose we copy to this empty disk a 100 MB file that cannot be compressed very much; let's suppose it can only be compressed at a ratio of 1.25 to 1. This means we will use up not 50 MB of real CVF space as we would expect from a file compressible at 2:1, but rather 80 MB (100 divided by 1.25). Here's what the disk will look like now:

As you can see, the 500 MB total for the CVF always remains the same (unless you resize the volume). But we have "lost" 60 MB from the compressed volume; it now has 840 MB free instead of the 900 MB we would expect after copying a 100 MB file to it. In reality, all that happened is that we weren't able to compress the file at the 2:1 ratio we expected.

Now let's copy another file to the compressed disk, let's say a 180 MB database file that will compress at a ratio of 4 to 1. (This is quite common with large database files, believe it or not.) This file will take only 45 MB of storage in the CVF, instead of the 90 MB that is "par" for a 2:1 volume. Here's what will happen:

After storing this file our totals have increased, because it used much less space than we "expected". In fact, our compressed volume is now "larger" than when we started! While this is an extreme example, this shows why the free space on a compressed disk tends to move around. The most common situation is when someone sets up a 1000 MB compressed disk and starts copying huge ZIP files to it. Guess what--ZIP files are already compressed and cannot be compressed further. As soon as you copy about 500 MB of ZIP files to a compressed volume of that size, it will be full.

Next: Slack Reduction Using Volume Compression