VMware Private AI Foundation with NVIDIA Now GA

Broadcom has recently announced the general availability of the VMware Private AI Foundation with NVIDIA, marking a significant enhancement in VMware Cloud Foundation’s capabilities. This addition enriches the product suite and empowers enterprises to leverage AI more effectively within their existing infrastructure.

Broadcom has recently announced the general availability of the VMware Private AI Foundation with NVIDIA, marking a significant enhancement in VMware Cloud Foundation’s capabilities. This addition enriches the product suite and empowers enterprises to leverage AI more effectively within their existing infrastructure.

VMware Private AI Foundation with NVIDIA is designed as an add-on for VMware Cloud Foundation. It allows enterprises to import pre-trained AI models into their data centers. Once integrated, these models can be fine-tuned using proprietary data or utilized in advanced Retrieval-Augmented Generation designs for specific business applications. This blend of flexibility and power is pivotal for businesses aiming to enhance operational efficiency and innovation.

Deeper GPU Monitoring and Operational Insights

From its initial release in March 2024, VMware has continually enhanced the system’s GPU monitoring capabilities. At its general availability, VMware introduced two new dashboards that provide comprehensive views of GPU usage across server clusters, crucial for maintaining performance and efficiency:

• Summary Dashboard: This dashboard provides a quick overview of GPU-equipped clusters, highlighting critical metrics like GPU memory, which is vital for performance scaling.
• Detailed Metrics Dashboard: This dashboard offers insights into GPU temperature, memory usage, and compute utilization, enabling proactive resource management and avoiding overheating risks.

Deep Dive into GPU Analytics

The newly released dashboards allow administrators to dig into the specifics of GPU performance across different clusters and servers. Each panel is color-coded to highlight critical thresholds for temperature, memory usage, and compute utilization, helping IT staff quickly identify and address potential issues. Users can select individual GPU tiles for deeper analysis to reveal detailed topology and operational data, ensuring precise troubleshooting and maintenance.

Automation with PowerCLI Scripts

To streamline the deployment and integration of the VMware Private AI Foundation with NVIDIA, VMware’s engineering team has developed a series of PowerCLI scripts. These scripts are essential tools for setting up the necessary infrastructure to support this sophisticated AI platform:

• Infrastructure Preparation: Scripts assist in creating VCF workload domains, setting up NVIDIA vGPU Managers, and establishing NSX Edge Clusters.
• Enabling AI Workloads: This section includes steps for activating the Kubernetes Supervisor Cluster within the Workload Domain, which is crucial for managing AI operations and deployments.

A critical component of the VMware Private AI Foundation with NVIDIA is its support for NVIDIA AI Enterprise containers and microservices. These include the NVIDIA NIM inference microservice and NVIDIA NeMo Retriever Microservice, which are central to deploying large language models and Retrieval-Augmented Generation applications. The PowerCLI scripts facilitate the automatic download and deployment of these services onto a Deep Learning VM during the initial setup.

The general availability of VMware Private AI Foundation with NVIDIA introduces robust tools and capabilities for enterprises seeking to enhance their data science and AI applications. The combination of advanced GPU monitoring, detailed operational insights, and powerful automation scripts simplifies the deployment of complex AI models, ensuring that enterprises can quickly and effectively integrate these technologies into their operational framework. This release signifies a substantial advancement in VMware’s commitment to integrating cutting-edge technology into its cloud offerings, providing enterprises with the tools they need to succeed in the evolving digital landscape.

Find more information in this VMware Private AI Foundation with NVIDIA solution brief.

Engage with StorageReview

Newsletter | YouTube | Podcast iTunes/Spotify | Instagram | Twitter | TikTok | RSS Feed