At the Open Compute Project Summit today Facebook is announcing its flexible NVMe JBOF (just a bunch of flash), Lightning. Facebook is introducing Lightning as a flash building block, or flash sled. Lightning will allow data centers to better scale out flash capacity across multiple applications while tuning the compute-to-storage ratio. Facebook is contributing this new JBOF to the Open Compute Project.
Facebook has been big on using flash for some time now. They use it for cache, database applications, and boot drives. While flash gives them the performance they are looking for it has a few drawbacks such as not scaling effectively. Flash density doubles roughly every year and a half making efficient scaling an issue. Facebook has been looking for ways to minimize the number of hardware building blocks while maximizing the total amount of flash available for given applications. Facebook saw disaggregating the hardware and software components as a possible solution to improving its operational efficiency. Enter their JBOF, Lightning.
Lightning provides a PCIe gen 3 connection from end to end (CPU to SSD). In order to provide a faster time to market, maintain a common look, enable modularity in PCIe switch solutions, and enable flexibility in SSD form factors Facebook is leveraging the existing Open Vault (Knox) SAS JBOD infrastructure. Facebook had to use a certain topology to implement the new JBOF (pictured above) and new boards:
Benefits include:
While an all NVMe sounds like a wonderful idea there are some concerns that need to be address before they can be implemented into data centers everywhere. NMVe can’t be hot-plugged or hot-added in the same fashion SAS drives can be. Currently PCIe hot-plugging is complicated. PCIe does not have an in-band enclosure and chassis management scheme like the SAS does, making management tricky. Part of the layout and new boards deals with retaining signal integrity. As opposed to using external PCIe cables, Facebook chose mini-SAS HD cables (SFF-8644). The cables use a full complement of PCIe side-band signals and a USB connection for an out-of-band management connection. And Facebook is dealing with power consumption of NVMe, a 2.5” NVMe SSD can use as much as 25W. Lightning is limiting the power to 14W per slot.
Lightning is designed to be a flexible, scalable solution for flash. It supports multiple SSD form factors and multiple head nodes while using a power level that makes sense for its targeted IOPS/TB. Adding this solution to OCP should bolster the NVMe ecosystem and accelerate the adoption of NVMe SSDs.
iXsystems has launched the TrueNAS Enterprise H-Series platforms, designed to give organizations ultimate performance. The H10 model is now available,…
Hannover Messe 2024 represents a significant event in the global industrial sector, serving as the world's largest industrial trade fair.…
The IBM Storage Assurance program offers access to the latest FlashSystem hardware and software, supporting investment protection from day one.…
Proxmox Backup Server 3.2 has been released - open-source solution designed for backup of VMs, containers, and physical hosts. (more…)
IBM has unveiled the FlashSystem 5300, setting a new standard for entry-level all-flash storage systems by providing impressive performance, high…
Proxmox Server Solutions has released the latest update to their server virtualization management platform, Proxmox VE 8.2. (more…)