--- description: "NVIDIA NVHBM puts the memory controller in the HBM base die: 30% more bandwidth, 15% lower power, 25% more compute area. Trainium4 first." title: NVIDIA NVHBM Moves the Memory Controller Into the HBM Stack, With Amazon's Trainium4 First in Line image: https://www.storagereview.com/wp-content/uploads/2026/08/gpu-architecture-corp-blog-nvhbm-1280x680-5547600.jpg --- [![Storage Review]()](https://storagereview.com/) --- [Facebook](https://www.facebook.com/@storagereview) [X (Twitter)](https://twitter.twitter.com/storagereview) [Instagram](https://www.instagram.com/storagereview/) [LinkedIn](https://www.linkedin.com/company/storagereview-com) [YouTube](https://www.youtube.com/user/storagereview?sub_confirmation=1) [Email](mailto:info@storagereview.com) [Spotify](https://open.spotify.com/show/1y6VnznABhHeOSMOmbDTz0) [Reddit](https://www.reddit.com/r/StorageReview/) [Discord](https://discord.gg/TwMHb4azdC) [RSS Feed](https://www.storagereview.com/rss.xml) [TikTok](https://www.tiktok.com/@storagereview) [Facebook](https://www.facebook.com/@storagereview) [X (Twitter)](https://twitter.com/storagereview) [Instagram](https://www.instagram.com/storagereview/) [LinkedIn](https://www.linkedin.com/company/storagereview-com) [YouTube](https://www.youtube.com/user/storagereview?sub_confirmation=1) [Email](mailto:info@storagereview.com) [Spotify](https://open.spotify.com/show/1y6VnznABhHeOSMOmbDTz0) [Reddit](https://www.reddit.com/r/StorageReview/) [Discord](https://discord.gg/TwMHb4azdC) [RSS Feed](https://www.storagereview.com/rss.xml) [TikTok](https://www.tiktok.com/@storagereview) [![StorageReview.com]()](https://www.storagereview.com) ≡ Menu - [Home](https://www.storagereview.com/) - [Storage Reviews](https://www.storagereview.com/review) - [End User Reviews](https://www.storagereview.com/consumer) - [Enterprise Reviews](https://www.storagereview.com/enterprise) - [Ubiquiti Reviews](https://www.storagereview.com/best/ubiquiti-reviews) - [SR Merch](https://store.storagereview.com) - [Leaderboards](https://www.storagereview.com/best) - [Best Storage Arrays](https://www.storagereview.com/best/storage-arrays) - [Best Enterprise SSDs](https://www.storagereview.com/best/enterprise-ssds) - [Best Servers](https://www.storagereview.com/best/servers) - [Best Enterprise GPUs](https://www.storagereview.com/best/enterprise-gpus) - [Best Desktops for Local AI](https://www.storagereview.com/best/desktops-local-ai) - [Best Laptops for Local AI](https://www.storagereview.com/best/laptops-local-ai) - [Best Local LLM Tools](https://www.storagereview.com/best/local-llm-tools) - [Agentic AI Hardware](https://www.storagereview.com/best/agentic-ai-hardware) - [Best SSDs & Hard Drives](https://www.storagereview.com/best_drives) - [Best Portable SSDs](https://www.storagereview.com/best/portable-ssds) - [Best Business Laptops](https://www.storagereview.com/best/business-laptops) - [Best Mobile Workstations](https://www.storagereview.com/best/mobile-workstations) - [Best Desktop Workstations](https://www.storagereview.com/best/desktop-workstations) - [Best Mini PCs](https://www.storagereview.com/best/mini-pcs) - [Best Laptop Battery Life](https://www.storagereview.com/best/laptop-battery-life) - [Storage Reference Guide](https://www.storagereview.com/storage-reference-guide) - [About SR](https://www.storagereview.com/about-storagereview) - [StorageReview.com Sweepstakes Rules and Regulations](https://www.storagereview.com/storagereview-com-sweepstakes-rules-and-regulations) Search [Home](https://www.storagereview.com/) » [News](https://www.storagereview.com/news) » NVIDIA NVHBM Moves the Memory Controller Into the HBM Stack, With Amazon’s Trainium4 First in Line # NVIDIA NVHBM Moves the Memory Controller Into the HBM Stack, With Amazon’s Trainium4 First in Line by Harold Fritts on August 27, 2026 [AI](https://www.storagereview.com/enterprise/ai)  ◇  [Enterprise](https://www.storagereview.com/enterprise) Hyperscalers and AI-focused infrastructure operators are increasingly developing proprietary AI accelerators and XPUs to keep pace with large language models and complex reasoning workloads. To address the physical, memory, and networking challenges of deploying proprietary silicon at scale, NVIDIA has detailed its [NVLink Fusion interconnect and NVHBM custom base-die memory architecture](https://blogs.nvidia.com/blog/nvlink-fusion-nvhbm-custom-high-bandwidth-memory/). The unified portfolio is designed to bridge custom silicon with NVIDIA’s existing scale-up networking and MGX rack-scale platforms. The first taker is Amazon’s Annapurna Labs, whose NVHBM plans surfaced alongside the [2 million GPU expansion of the AWS partnership](https://www.storagereview.com/news/aws-and-nvidia-to-deploy-2-million-more-gpus-in-2027-2028-bringing-vera-cpus-and-custom-nvhbm-to-trainium); this announcement supplies the architecture behind that tie-up and names Trainium4 as the first Trainium generation with NVLink Fusion support. NVIDIA is also establishing a standard NVHBM implementation available from multiple memory providers, which it says reduces the engineering effort of integrating and qualifying memory across suppliers. “NVHBM represents a new architectural approach to advancing high-bandwidth memory performance and efficiency,” said Nafea Bshara, vice president of Annapurna Labs at Amazon. “We look forward to this technology collaboration to benefit future AWS infrastructure designs.” ![NVIDIA diagram comparing a custom XPU with NVHBM, showing a narrower memory interface and larger compute die area, against the same XPU with standard HBM]() Deploying custom XPUs across high-density data centers requires balancing compute logic, power distribution, thermal overhead, and high-bandwidth memory. Integrating and qualifying leading-edge memory stacks often creates significant packaging and development bottlenecks. Through NVLink Fusion and NVHBM, designers can leverage pre-validated base dies and interface IP, reducing integration complexity and accelerating time-to-market. ## Memory Bandwidth and Latency Optimization Modern AI workloads, particularly large-model inference and agentic pipelines, place heavy demands on memory throughput for reading model weights, activations, and key-value (KV) cache entries. NVHBM provides a custom base die developed in coordination with memory manufacturers, delivering up to 30 percent higher memory bandwidth per stack compared to the standard JEDEC HBM4E specification. | Feature | NVHBM benefit | | --- | --- | | Bandwidth | Up to 30% more memory bandwidth compared with standard HBM4e | | Area | More efficient interface connections allow up to 25% more compute die area for additional XPU capabilities | | Power | Up to 15% lower HBM power usage compared with standard HBM4e adds up to savings across thousands of XPUs | Table 1. NVHBM brings three main platform-level advantages to AI accelerator programs: higher memory bandwidth, more package and silicon area, and lower HBM power usage This bandwidth uplift minimizes compute engine starvation during memory-bound phases of execution. By accelerating data transfers between the HBM stacks and onboard compute logic, accelerators can sustain higher utilization and improve per-user token generation rates during dense inference workloads. ## Die Area Reallocation and PHY Optimization Silicon real estate on modern accelerator packages is strictly bounded, requiring designers to balance space between compute ALUs, matrix engines, on-chip SRAM, and peripheral interfaces. Standard HBM architectures require wide physical interfaces that consume substantial layout area. ![NVIDIA render of a custom XPU package with NVHBM memory stacks flanking the compute dies]() NVHBM addresses this by moving the memory controller directly into the 3D HBM stack and utilizing a custom physical layer (PHY). This approach yields up to a 67 percent reduction in PHY and peripheral support area compared to standard HBM4E implementations, while simplifying interposer trace routing to reclaim up to 80 percent more usable silicon across the layout. The narrower memory interface frees up package real estate, allowing designers to expand the central AI compute die by up to 30 percent to add more matrix cores, vector units, or cache hierarchy within a fixed physical envelope. ## Power Efficiency and Facility-Scale Headroom Power distribution remains one of the tightest constraints in hyperscale data center operations, directly impacting accelerator thermal design and cooling requirements. NVHBM reduces HBM power consumption by 15 percent compared to standard HBM4E. At the silicon level, reduced memory power lowers thermal pressure, providing additional electrical headroom to drive compute cores at higher sustained frequencies. At facility scale, these power savings multiply significantly. Across a theoretical 1-gigawatt data center deploying 2,000-watt XPUs, the reduced memory power profile can free up enough thermal and electrical capacity to support up to 15,000 additional XPUs. ## Rack-Scale Integration via NVLink Fusion While NVHBM optimizes performance at the individual-package level, NVLink Fusion serves as the system-level bridge. Utilizing dedicated NVLink Fusion chiplets, custom XPUs can attach directly to the [sixth-generation NVLink scale-up fabric](https://www.storagereview.com/news/nvidia-groq-3-lpx-enters-full-production-3400-tokens-per-second-at-100k-context-256-lp30s-per-rack), uniting all accelerators within a rack into a single coherent memory and compute domain. This scale-up fabric is critical for advanced parallelization schemes such as expert parallelism (EP) and WideEP, in which distributed mixture-of-experts models require low-latency synchronization of activations and hidden states across multiple physical devices. Custom processors can also link to host CPUs via NVLink-C2C. By integrating semi-custom silicon into the broader NVIDIA hardware and software ecosystem, NVLink Fusion enables operators to deploy heterogeneous environments that mix custom XPUs with standard NVIDIA GPUs under a standardized MGX rack architecture, simplifying operations and workload provisioning. **Engage with StorageReview** [Newsletter](https://www.storagereview.com/storage_newsletter) |  [YouTube](https://www.youtube.com/user/StorageReview "Opens in a new window") | Podcast  [iTunes](https://podcasts.apple.com/gb/podcast/storagereview-com-storage-reviews/id1060681115 "Opens in a new window")/ [Spotify](https://open.spotify.com/show/1y6VnznABhHeOSMOmbDTz0 "Opens in a new window") |  [Instagram](https://www.instagram.com/storagereview/ "Opens in a new window") |  [Twitter](https://twitter.com/storagereview "Opens in a new window") |  [TikTok](https://www.tiktok.com/@storagereview? "Opens in a new window") |  [RSS Feed](https://www.storagereview.com/rss.xml) ![]() ### Harold Fritts I have been in the tech industry since IBM created Selectric. My background, though, is writing. So I decided to get out of the pre-sales biz and return to my roots, doing a bit of writing but still being involved in technology. Previous post: [AWS and NVIDIA to Deploy 2 Million More GPUs in 2027-2028, Bringing Vera CPUs and Custom NVHBM to Trainium](https://www.storagereview.com/news/aws-and-nvidia-to-deploy-2-million-more-gpus-in-2027-2028-bringing-vera-cpus-and-custom-nvhbm-to-trainium) Next post: [IBM DS8000 Release 10.2 Brings FCM5 in EDSFF, Doubled Back-End Throughput, and Automated Ransomware Containment](https://www.storagereview.com/news/ibm-ds8000-release-10-2-brings-fcm5-in-edsff-doubled-back-end-throughput-and-automated-ransomware-containment) Trusted Vendors Products and solutions from our affiliate partners: - [ ![Ubiquiti Logo]() Ubiquiti ](https://store.ui.com/us/en?a_aid=StorageReview "Opens in a new window") - [ ![Newegg Logo]() Newegg ](https://click.linksynergy.com/fs-bin/click?id=g5terNMDhz0&offerid=1207190.18&subid=0&type=4 "Opens in a new window") - [ ![Amazon Logo]() Amazon ](https://amzn.to/4fLJUEW "Opens in a new window") Newsletter Subscribe to the StorageReview newsletter to stay up to date on the latest news and reviews. We promise no spam! 1 Leave this field empty if you’re human: ## Advertisement Content Categories [Facebook](https://www.facebook.com/@storagereview) [X](https://twitter.com/storagereview) [Instagram](https://www.instagram.com/storagereview/) [LinkedIn](https://www.linkedin.com/company/storagereview-com) [YouTube](https://www.youtube.com/user/storagereview?sub_confirmation=1) [Email](mailto:info@storagereview.com) [Spotify](https://open.spotify.com/show/1y6VnznABhHeOSMOmbDTz0) [Reddit](https://www.reddit.com/r/StorageReview/) [Discord](https://discord.gg/TwMHb4azdC) [RSS](https://www.storagereview.com/rss.xml) [TikTok](https://www.tiktok.com/@storagereview) Copyright © 1998-2025 Flying Pig Ventures, LLC Cincinnati, Ohio. All rights reserved. Manage your privacy To provide the best experiences, we and our partners use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us and our partners to process personal data such as browsing behavior or unique IDs on this site and show (non-) personalized ads. Not consenting or withdrawing consent, may adversely affect certain features and functions. Click below to consent to the above or make granular choices. Your choices will be applied to this site only. You can change your settings at any time, including withdrawing your consent, by using the toggles on the Cookie Policy, or by clicking on the manage consent button at the bottom of the screen. Functional Functional Always active The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network. Preferences Preferences The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user. Statistics Statistics The technical storage or access that is used exclusively for statistical purposes. The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you. Marketing Marketing The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes. Statistics Marketing Features Always active Special features Special purposes Always active - Manage options - Manage services - Manage {vendor\_count} vendors - [Read more about these purposes](https://cookiedatabase.org/tcf/purposes/) Accept Deny Manage options Save preferences Manage options - {title} - {title} - {title} Manage your privacy To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions. Functional Functional Always active The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network. Preferences Preferences The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user. Statistics Statistics The technical storage or access that is used exclusively for statistical purposes. The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you. Marketing Marketing The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes. Statistics Marketing Features Always active Special features Special purposes Always active - Manage options - Manage services - Manage {vendor\_count} vendors - [Read more about these purposes](https://cookiedatabase.org/tcf/purposes/) Accept Deny Manage options Save preferences Manage options - {title} - {title} - {title} Manage consent Manage consent ```json {"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.storagereview.com\/news\/nvidia-nvhbm-moves-the-memory-controller-into-the-hbm-stack-with-amazons-trainium4-first-in-line","url":"https:\/\/www.storagereview.com\/news\/nvidia-nvhbm-moves-the-memory-controller-into-the-hbm-stack-with-amazons-trainium4-first-in-line","name":"NVIDIA NVHBM Moves the Memory Controller Into the HBM Stack, With Amazon's Trainium4 First in Line - StorageReview.com","isPartOf":{"@id":"https:\/\/www.storagereview.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.storagereview.com\/news\/nvidia-nvhbm-moves-the-memory-controller-into-the-hbm-stack-with-amazons-trainium4-first-in-line#primaryimage"},"image":{"@id":"https:\/\/www.storagereview.com\/news\/nvidia-nvhbm-moves-the-memory-controller-into-the-hbm-stack-with-amazons-trainium4-first-in-line#primaryimage"},"thumbnailUrl":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2026\/08\/gpu-architecture-corp-blog-nvhbm-1280x680-5547600.jpg","datePublished":"2026-08-27T16:00:06+00:00","dateModified":"2026-08-27T16:16:35+00:00","description":"NVIDIA NVHBM puts the memory controller in the HBM base die: 30% more bandwidth, 15% lower power, 25% more compute area. Trainium4 first.","breadcrumb":{"@id":"https:\/\/www.storagereview.com\/news\/nvidia-nvhbm-moves-the-memory-controller-into-the-hbm-stack-with-amazons-trainium4-first-in-line#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.storagereview.com\/news\/nvidia-nvhbm-moves-the-memory-controller-into-the-hbm-stack-with-amazons-trainium4-first-in-line"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.storagereview.com\/news\/nvidia-nvhbm-moves-the-memory-controller-into-the-hbm-stack-with-amazons-trainium4-first-in-line#primaryimage","url":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2026\/08\/gpu-architecture-corp-blog-nvhbm-1280x680-5547600.jpg","contentUrl":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2026\/08\/gpu-architecture-corp-blog-nvhbm-1280x680-5547600.jpg","width":1280,"height":680,"caption":"NVIDIA render of a custom XPU package with NVHBM memory stacks flanking the compute dies"},{"@type":"BreadcrumbList","@id":"https:\/\/www.storagereview.com\/news\/nvidia-nvhbm-moves-the-memory-controller-into-the-hbm-stack-with-amazons-trainium4-first-in-line#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.storagereview.com\/"},{"@type":"ListItem","position":2,"name":"News","item":"https:\/\/www.storagereview.com\/news"},{"@type":"ListItem","position":3,"name":"NVIDIA NVHBM Moves the Memory Controller Into the HBM Stack, With Amazon’s Trainium4 First in Line"}]},{"@type":"WebSite","@id":"https:\/\/www.storagereview.com\/#website","url":"https:\/\/www.storagereview.com\/","name":"StorageReview.com","description":"StorageReview.com is a leading provider of news and reviews throughout the entire IT stack - from the datacenter to the edge, and all points in between.","publisher":{"@id":"https:\/\/www.storagereview.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.storagereview.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.storagereview.com\/#organization","name":"StorageReview.com","url":"https:\/\/www.storagereview.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.storagereview.com\/#\/schema\/logo\/image\/","url":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2020\/02\/Storage-Reviews-2-2.png","contentUrl":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2020\/02\/Storage-Reviews-2-2.png","width":344,"height":61,"caption":"StorageReview.com"},"image":{"@id":"https:\/\/www.storagereview.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/storagereview","http:\/\/youtube.com\/user\/storagereview"]}]} ```