--- description: At the Open Compute Project Summit today Facebook is announcing its flexible NVMe JBOF (just a bunch of flash), Lightning. Facebook is introducing... title: Facebook Introduces A Flexible NVMe JBOF image: https://www.storagereview.com/wp-content/uploads/2015/06/Facebook_logo_square.png --- [![Storage Review]()](https://storagereview.com/) --- [Facebook](https://www.facebook.com/@storagereview) [X (Twitter)](https://twitter.twitter.com/storagereview) [Instagram](https://www.instagram.com/storagereview/) [LinkedIn](https://www.linkedin.com/company/storagereview-com) [YouTube](https://www.youtube.com/user/storagereview?sub_confirmation=1) [Email](mailto:info@storagereview.com) [Spotify](https://open.spotify.com/show/1y6VnznABhHeOSMOmbDTz0) [Reddit](https://www.reddit.com/r/StorageReview/) [Discord](https://discord.gg/TwMHb4azdC) [RSS Feed](https://www.storagereview.com/rss.xml) [TikTok](https://www.tiktok.com/@storagereview) [Facebook](https://www.facebook.com/@storagereview) [X (Twitter)](https://twitter.com/storagereview) [Instagram](https://www.instagram.com/storagereview/) [LinkedIn](https://www.linkedin.com/company/storagereview-com) [YouTube](https://www.youtube.com/user/storagereview?sub_confirmation=1) [Email](mailto:info@storagereview.com) [Spotify](https://open.spotify.com/show/1y6VnznABhHeOSMOmbDTz0) [Reddit](https://www.reddit.com/r/StorageReview/) [Discord](https://discord.gg/TwMHb4azdC) [RSS Feed](https://www.storagereview.com/rss.xml) [TikTok](https://www.tiktok.com/@storagereview) [![StorageReview.com](https://www.storagereview.com/wp-content/uploads/2026/07/Storage-Review-2x.png)](https://www.storagereview.com) ≡ Menu - [Home](https://www.storagereview.com/) - [Storage Reviews](https://www.storagereview.com/review) - [End User Reviews](https://www.storagereview.com/consumer) - [Enterprise Reviews](https://www.storagereview.com/enterprise) - [Ubiquiti Reviews](https://www.storagereview.com/best/ubiquiti-reviews) - [SR Merch](https://store.storagereview.com) - [Leaderboards](https://www.storagereview.com/best) - [Best Storage Arrays](https://www.storagereview.com/best/storage-arrays) - [Best Enterprise SSDs](https://www.storagereview.com/best/enterprise-ssds) - [Best Servers](https://www.storagereview.com/best/servers) - [Best Enterprise GPUs](https://www.storagereview.com/best/enterprise-gpus) - [Best Desktops for Local AI](https://www.storagereview.com/best/desktops-local-ai) - [Best Laptops for Local AI](https://www.storagereview.com/best/laptops-local-ai) - [Best Local LLM Tools](https://www.storagereview.com/best/local-llm-tools) - [Agentic AI Hardware](https://www.storagereview.com/best/agentic-ai-hardware) - [Best SSDs & Hard Drives](https://www.storagereview.com/best_drives) - [Best Portable SSDs](https://www.storagereview.com/best/portable-ssds) - [Best Business Laptops](https://www.storagereview.com/best/business-laptops) - [Best Mobile Workstations](https://www.storagereview.com/best/mobile-workstations) - [Best Desktop Workstations](https://www.storagereview.com/best/desktop-workstations) - [Best Mini PCs](https://www.storagereview.com/best/mini-pcs) - [Best Laptop Battery Life](https://www.storagereview.com/best/laptop-battery-life) - [Storage Reference Guide](https://www.storagereview.com/storage-reference-guide) - [About SR](https://www.storagereview.com/about-storagereview) - [StorageReview.com Sweepstakes Rules and Regulations](https://www.storagereview.com/storagereview-com-sweepstakes-rules-and-regulations) Search [Home](https://www.storagereview.com/) » [News](https://www.storagereview.com/news) » Facebook Introduces A Flexible NVMe JBOF # Facebook Introduces A Flexible NVMe JBOF by Adam Armstrong on March 9, 2016 At the Open Compute Project Summit today Facebook is announcing its flexible NVMe JBOF (just a bunch of flash), Lightning. Facebook is introducing Lightning as a flash building block, or flash sled. Lightning will allow data centers to better scale out flash capacity across multiple applications while tuning the compute-to-storage ratio. Facebook is contributing this new JBOF to the Open Compute Project. ![](https://www.storagereview.com/wp-content/uploads/2016/03/StorageReview-Facebook-Lightning.jpg) Facebook has been big on using flash for some time now. They use it for cache, database applications, and boot drives. While flash gives them the performance they are looking for it has a few drawbacks such as not scaling effectively. Flash density doubles roughly every year and a half making efficient scaling an issue. Facebook has been looking for ways to minimize the number of hardware building blocks while maximizing the total amount of flash available for given applications. Facebook saw disaggregating the hardware and software components as a possible solution to improving its operational efficiency. Enter their JBOF, Lightning. Lightning provides a PCIe gen 3 connection from end to end (CPU to SSD). In order to provide a faster time to market, maintain a common look, enable modularity in PCIe switch solutions, and enable flexibility in SSD form factors Facebook is leveraging the existing Open Vault (Knox) SAS JBOD infrastructure. Facebook had to use a certain topology to implement the new JBOF (pictured above) and new boards: - PCIe retimer card. This x16 PCIe gen 3 card is installed in the Leopard server, which is used as a head node for the JBOF. It is a simple, low-cost, low-power design to retransmit the PCIe signals over external mini-SAS HD (SFF-8644) cables of at least 2 meters. - PCIe Expansion Board (PEB). The PCIe switch, the Lightning BMC, and all the switch support circuitry are located on this board. One PEB is installed per tray of SSDs and replaces the SAS Expander Board (SEB) used in Knox. This allows us to use a common switch board for both trays and also allows us to easily design new or different versions (with next generation switches, for example) without modifying the rest of the infrastructure. Each PEB features up to 32x lanes of PCIe as the uplink(s) to the retimer in the head node and 60x lanes of PCIe to the SSDs. The PEB can be hot-swapped if it fails without affecting the other tray in the system. - PCIe Drive Plane Board (PDPB). The PDPB contains the 15x SFF-8639 (U.2) SSD connectors and support circuitry. Each SSD connector, or slot, is connected to 4x lanes of PCIe, PCIe clocks, PCIe reset, and additional side-band signals from the PCIe switch or switches on the PEB. Each SSD slot can also be bifurcated into 2x x2 ports, which allows us to double the number of SSDs per tray from 15x to 30x (from 30x per system to 60x per system) without requiring an additional layer of PCIe switching. **Benefits include:** - Lightning can support a variety of SSD form factors, including 2.5", M.2, and 3.5" SSDs. - Lightning will support surprise hot-add and surprise hot-removal of SSDs to make field replacements as simple and transparent as SAS JBODs. - Lightning will use an ASPEED AST2400 BMC chip running OpenBMC. - Lightning will support multiple switch configurations, which allows us to support different SSD configurations (for example, 15x x4 SSDs vs. 30x x2 SSDs) or different head node to SSD mappings without changing HW in any way. - Lightning will be capable of supporting up to four head nodes. By supporting multiple head nodes per tray, we can adjust the compute-to-storage ratio as needed simply by changing the switch configuration. While an all NVMe sounds like a wonderful idea there are some concerns that need to be address before they can be implemented into data centers everywhere. NMVe can’t be hot-plugged or hot-added in the same fashion SAS drives can be. Currently PCIe hot-plugging is complicated. PCIe does not have an in-band enclosure and chassis management scheme like the SAS does, making management tricky. Part of the layout and new boards deals with retaining signal integrity. As opposed to using external PCIe cables, Facebook chose mini-SAS HD cables (SFF-8644). The cables use a full complement of PCIe side-band signals and a USB connection for an out-of-band management connection. And Facebook is dealing with power consumption of NVMe, a 2.5” NVMe SSD can use as much as 25W. Lightning is limiting the power to 14W per slot. Lightning is designed to be a flexible, scalable solution for flash. It supports multiple SSD form factors and multiple head nodes while using a power level that makes sense for its targeted IOPS/TB. Adding this solution to OCP should bolster the NVMe ecosystem and accelerate the adoption of NVMe SSDs. [Engineering at Facebook](https://code.facebook.com) [Open Compute Project](http://www.opencompute.org) [Discuss this story](http://forums.storagereview.com/index.php/topic/38593-facebook-introduces-a-flexible-nvme-jbof/) [Sign up for the StorageReview newsletter](https://www.storagereview.com/storage_newsletter) **Engage with StorageReview** [Newsletter](https://www.storagereview.com/storage_newsletter) |  [YouTube](https://www.youtube.com/user/StorageReview "Opens in a new window") | Podcast  [iTunes](https://podcasts.apple.com/gb/podcast/storagereview-com-storage-reviews/id1060681115 "Opens in a new window")/ [Spotify](https://open.spotify.com/show/1y6VnznABhHeOSMOmbDTz0 "Opens in a new window") |  [Instagram](https://www.instagram.com/storagereview/ "Opens in a new window") |  [Twitter](https://twitter.com/storagereview "Opens in a new window") |  [TikTok](https://www.tiktok.com/@storagereview? "Opens in a new window") |  [RSS Feed](https://www.storagereview.com/rss.xml) ![]() ### Adam Armstrong Adam is the chief news editor for StorageReview.com, managing our internal and freelance content teams. Previous post: [Google Joins Open Compute Project](https://www.storagereview.com/news/google-joins-open-compute-project) Next post: [Facebook Focuses On More Efficient Front-End Servers](https://www.storagereview.com/news/facebook-focuses-on-more-efficient-front-end-servers) Trusted Vendors Products and solutions from our affiliate partners: - [ ![Ubiquiti Logo](https://www.storagereview.com/wp-content/uploads/2024/11/2-1.webp) Ubiquiti ](https://store.ui.com/us/en?a_aid=StorageReview "Opens in a new window") - [ ![Newegg Logo](https://www.storagereview.com/wp-content/uploads/2025/11/newegg.webp) Newegg ](https://click.linksynergy.com/fs-bin/click?id=g5terNMDhz0&offerid=1207190.18&subid=0&type=4 "Opens in a new window") - [ ![Amazon Logo](https://www.storagereview.com/wp-content/uploads/2025/11/amazon.webp) Amazon ](https://amzn.to/4fLJUEW "Opens in a new window") Newsletter Subscribe to the StorageReview newsletter to stay up to date on the latest news and reviews. We promise no spam! 1 Leave this field empty if you’re human: Content Categories [Facebook](https://www.facebook.com/@storagereview) [X](https://twitter.com/storagereview) [Instagram](https://www.instagram.com/storagereview/) [LinkedIn](https://www.linkedin.com/company/storagereview-com) [YouTube](https://www.youtube.com/user/storagereview?sub_confirmation=1) [Email](mailto:info@storagereview.com) [Spotify](https://open.spotify.com/show/1y6VnznABhHeOSMOmbDTz0) [Reddit](https://www.reddit.com/r/StorageReview/) [Discord](https://discord.gg/TwMHb4azdC) [RSS](https://www.storagereview.com/rss.xml) [TikTok](https://www.tiktok.com/@storagereview) Copyright © 1998-2025 Flying Pig Ventures, LLC Cincinnati, Ohio. All rights reserved. Manage your privacy To provide the best experiences, we and our partners use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us and our partners to process personal data such as browsing behavior or unique IDs on this site and show (non-) personalized ads. Not consenting or withdrawing consent, may adversely affect certain features and functions. Click below to consent to the above or make granular choices. Your choices will be applied to this site only. You can change your settings at any time, including withdrawing your consent, by using the toggles on the Cookie Policy, or by clicking on the manage consent button at the bottom of the screen. Functional Functional Always active The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network. Preferences Preferences The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user. Statistics Statistics The technical storage or access that is used exclusively for statistical purposes. The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you. Marketing Marketing The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes. Statistics Marketing Features Always active Special features Special purposes Always active - Manage options - Manage services - Manage {vendor\_count} vendors - [Read more about these purposes](https://cookiedatabase.org/tcf/purposes/) Accept Deny Manage options Save preferences Manage options - {title} - {title} - {title} Manage your privacy To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions. Functional Functional Always active The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network. Preferences Preferences The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user. Statistics Statistics The technical storage or access that is used exclusively for statistical purposes. The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you. Marketing Marketing The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes. Statistics Marketing Features Always active Special features Special purposes Always active - Manage options - Manage services - Manage {vendor\_count} vendors - [Read more about these purposes](https://cookiedatabase.org/tcf/purposes/) Accept Deny Manage options Save preferences Manage options - {title} - {title} - {title} Manage consent Manage consent ```json {"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.storagereview.com\/news\/facebook-introduces-a-flexible-nvme-jbof","url":"https:\/\/www.storagereview.com\/news\/facebook-introduces-a-flexible-nvme-jbof","name":"Facebook Introduces A Flexible NVMe JBOF - StorageReview.com","isPartOf":{"@id":"https:\/\/www.storagereview.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.storagereview.com\/news\/facebook-introduces-a-flexible-nvme-jbof#primaryimage"},"image":{"@id":"https:\/\/www.storagereview.com\/news\/facebook-introduces-a-flexible-nvme-jbof#primaryimage"},"thumbnailUrl":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2015\/06\/Facebook_logo_square.png","datePublished":"2016-03-09T20:05:47+00:00","breadcrumb":{"@id":"https:\/\/www.storagereview.com\/news\/facebook-introduces-a-flexible-nvme-jbof#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.storagereview.com\/news\/facebook-introduces-a-flexible-nvme-jbof"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.storagereview.com\/news\/facebook-introduces-a-flexible-nvme-jbof#primaryimage","url":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2015\/06\/Facebook_logo_square.png","contentUrl":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2015\/06\/Facebook_logo_square.png","width":480,"height":480},{"@type":"BreadcrumbList","@id":"https:\/\/www.storagereview.com\/news\/facebook-introduces-a-flexible-nvme-jbof#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.storagereview.com\/"},{"@type":"ListItem","position":2,"name":"News","item":"https:\/\/www.storagereview.com\/news"},{"@type":"ListItem","position":3,"name":"Facebook Introduces A Flexible NVMe JBOF"}]},{"@type":"WebSite","@id":"https:\/\/www.storagereview.com\/#website","url":"https:\/\/www.storagereview.com\/","name":"StorageReview.com","description":"StorageReview.com is a leading provider of news and reviews throughout the entire IT stack - from the datacenter to the edge, and all points in between.","publisher":{"@id":"https:\/\/www.storagereview.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.storagereview.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.storagereview.com\/#organization","name":"StorageReview.com","url":"https:\/\/www.storagereview.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.storagereview.com\/#\/schema\/logo\/image\/","url":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2020\/02\/Storage-Reviews-2-2.png","contentUrl":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2020\/02\/Storage-Reviews-2-2.png","width":344,"height":61,"caption":"StorageReview.com"},"image":{"@id":"https:\/\/www.storagereview.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/storagereview","http:\/\/youtube.com\/user\/storagereview"]}]} ```