Our current AI system reviews center on inference serving in the lab, with open-weight models from the Llama, Qwen, DeepSeek, and gpt-oss families running through vLLM or SGLang and measured on output throughput and first-token latency. Coverage runs from the edge to the full GPU node: the NVIDIA RTX PRO 4500 Server Edition delivered a median 2.7x the output tokens per second of the L4 it was built to replace inside an HPE ProLiant DL145 Gen11, for about 105W more at the wall; the Dell, GIGABYTE, and HP versions of DGX Spark ran distributed inference as two-node clusters over the 200 Gb fabric; and Supermicro JumpStart gave us hands-on time with an NVIDIA HGX B200 and an AMD Instinct MI350X system. Storage gets the same treatment because the data path decides how busy those GPUs stay, so we measure GPUDirect Storage throughput with GDSIO and test KV cache offload to flash for long-context and agentic workloads.
If you are sizing hardware for AI, the Best Servers page names our current AI inference and edge picks on the data from these reviews, Best Enterprise SSDs covers the drives that keep the GPUs fed, and Best Storage Arrays covers the AI storage platforms behind training clusters and neoclouds.
The NVIDIA RTX PRO 4500 Blackwell Server Edition is the card NVIDIA built primarily to replace the L4 in servers that live outside the data center: single-slot, PCIe Gen5, passively cooled, 165W, with 32GB of GDDR7 and the full Blackwell feature set, including FP4 Tensor Cores and Multi-Instance GPU. HPE sent us a ProLiant DL145
Enterprise AI infrastructure has shifted from optimizing training models to serving them, and that changes the economics. Training is a capital project with an endpoint. Inference is a production workload that runs as long as the service is live, with output measured in tokens. This is the tokenomics problem now facing AI operators: once the
Two things tend to come up first whenever the conversation turns to the NVIDIA DGX Spark. The first is the headline spec: 128 GB of unified memory in a roughly $4,000 desktop box, a number that would have seemed implausible to put on an engineer’s desk even two years ago. The second is the 200
When we published our story on Oregon State University’s plankton imaging research last November, the headline was the science: AI-accelerated infrastructure aboard research vessels, processing terabytes of ocean data in near real-time before the ship ever reached port. But something else happened quietly in the weeks that followed. Word spread across campus about what a
Supermicro’s JumpStart program has established itself as one of the more useful tools in the pre-purchase evaluation toolkit for AI infrastructure. Rather than a scripted demo in a shared environment, JumpStart gives qualified users free, time-boxed, bare-metal access to real production servers via SSH, IPMI, and VNC, enabling them to run workloads on actual hardware.
We spent a week testing an HGX B200 through Supermicro JumpStart to see how remote bare-metal access speeds AI platform decisions.
Oregon State is advancing AI ocean research with GPU, compute, and dense NVMe to analyze plankton data in real-time at sea.
The L40S is especially valuable for synthetic data generation, multimodal AI development, and Omniverse applications that require both computing and graphics performance.
Pliops XDP LightningAI boosts LLM inference by offloading KV cache, enabling faster, scalable AI with NVIDIA Dynamo integration.
DUG Nomad mobile data centers deliver immersion-cooled AI and HPC capabilities at the edge with Hypertec servers and Solidigm SSDs.
Amazon