One of the little known or little explored secrets with servers running VMware ESXi 5.x is they can be booted off of USB or SD flash media. In such a scenario, enterprises could dump HBAs or RAID cards and the hard drives or SSDs that are used to boot the servers. Going this route yields a CAPEX benefit for new systems, the cost of flash media for boot is insignificant compared to the drives and storage adapter cards required to spin up a server. There are also gains in OPEX; the implications of moving to USB or SD flash media have a big advantage when considering the power required for a small flash media drive for boot vs. the more common practice of adapters and drives.
In this piece we specifically look at the impact that flash media can have in data center operations. VMware actually has a knowledge base article on how to deploy USB or SD flash media as a boot device for ESXi 5.x and most server vendors offer configurations with SD media installed for operations such as this. The VMware KB piece is pretty straight forward and recommends a deployment of a drive no less than 8GB in capacity, so there's room for a scratch partition on the drive. VMware goes on to suggest a 16GB or larger drive is ideal so that the extra flash can be used for wear leveling to prolong the live of the drive. Given the cost delta between 8GB and 16GB, there's little reason to not opt for the 16GB drives. The only other consideration is card quality, which is why we've selected SanDisk for this project. SanDisk consistently proves their leadership position in portable flash media, routinely besting all others in performance and warranty. While throughput is less of a concern here, most servers are using USB 2.0, the heritage is important, as even data centers relying on clustering for virtualized servers appreciate any improvement in component failure rates.
The SanDisk Cruzer Fit tucked into the RD630 motherboard
We're using ten of the SanDisk Cruzer Fit USB Flash Drives for this test. The Cruzer Fit is a low-profile, unobtrusive flash drive that can hide away on internal server USB ports, or sit close to the server if needed to be mounted in front. It is stripped of any embellishments (other than its general ergonomic design) and is as minimalistic as it gets for flash drives. The 16GB version retails for a little more than $10 and scales up to 64GB with a retail of under $60.
How to Install ESXi on Portable Flash Media
VMware offers a step-by-step set of instructions in the KB article on how to accomplish this. We've simplified the steps some, opting for a virtual optical drive rather than burning and booting from optical media. Our process for installation is as follows:
After the system boots, users will be presented with a warning that there is no local datastore and logs are being stored on non-persistent storage. To fix these issues, attach your networked shared storage of choice, such as iSCSI, FC or NFS storage.
Once one or more datastores are attached to the ESXi host, to solve the log warning, go into the advanced settings and assign a location for the system logs. In our case we used our datastore "Storage" for this purpose.
USB Flash Drive as ESXi Boot Drive
In testing we started out by deploying a Lenovo ThinkServer RD630 in a variety of different configurations and testing power consumption at idle via Eaton Advanced ePDU to get baseline stats. These figures were polled after VMware ESXI 5.1 was installed, booted, and sitting idle in the "Balanced" power mode.
We leveraged three HDD and one SSD configurations in RAID1 connected through the RD630's stock LSI 9260-8i RAID card with battery backup. The last configuration was the USB thumbdrive alone with the RAID card removed.
In the five configurations tested, we measured power savings ranging from 21 to 35W, which translates into significant power savings over the span of each year per server. With an average cost of $0.12/kWh, power savings ranged from $22 to just under $37 per year. When those savings are multiplied across many servers, power savings add up. In addition to sheer power savings though, the reduction in components also leads to reduced heat and potential savings in cooling.
Beyond the OPEX savings there are huge savings on the CAPEX side. In just the scenarios tested, the RAID card and base HDDs or SSDs range from $800 to as much as $1,500 when configuring servers from popular tier 1 brands. This means that when deploying a handful of servers, either businesses can take advantage of the savings by spending less, or the same spend can go further. This could mean another server node or two for a cluster, or funds could be allocated to invest further in CPUs and memory to support the virtualized environment.
This plan of attack may not be for everyone, the traditional RAID card or HBA paired with drives will still dominate boot duty for most. At scale though, these expenses add up, not to mention the OPEX benefits of reducing power consumption in the highly virtualized data center. For a $10 per server investment though, it's hard to figure another way a Hamilton could be better spent.
The UGREEN DXP480T Plus offers an alluring blend of portability and performance in a tiny body with 4 M.2 NVMe…
TrueNAS has gained traction in the self-host and homelab communities for several reasons. One primary reason is that it's free…
The Intel Data Center GPU Flex Series 170 ia tantalizing for organizations that want to provide their VDI users with…
The UGREEN DXP6800 Pro is an exciting addition to the prebuilt NAS segment and stands out as a sophisticated solution…
Overall, the Latitude 9450 2-in-1 gets our strong recommendation for a skinny high-end business 2-in-1 convertible. (more…)
For the 2024 eclipse we needed fast, reliable storage that could handle a few bumps and bruises along the way…