--- description: Optimize LLM inference with Pliops and vLLM. Enhance performance, reduce costs, and scale AI workloads with KV cache acceleration. title: "Pliops and vLLM: Smarter KV Caching for LLM Inference" image: https://www.storagereview.com/wp-content/uploads/2022/07/StorageReview-Pliops-XDP-2.jpg --- [![Storage Review]()](https://storagereview.com/) --- [Facebook](https://www.facebook.com/@storagereview) [X (Twitter)](https://twitter.twitter.com/storagereview) [Instagram](https://www.instagram.com/storagereview/) [LinkedIn](https://www.linkedin.com/company/storagereview-com) [YouTube](https://www.youtube.com/user/storagereview?sub_confirmation=1) [Email](mailto:info@storagereview.com) [Spotify](https://open.spotify.com/show/1y6VnznABhHeOSMOmbDTz0) [Reddit](https://www.reddit.com/r/StorageReview/) [Discord](https://discord.gg/TwMHb4azdC) [RSS Feed](https://www.storagereview.com/rss.xml) [TikTok](https://www.tiktok.com/@storagereview) [Facebook](https://www.facebook.com/@storagereview) [X (Twitter)](https://twitter.com/storagereview) [Instagram](https://www.instagram.com/storagereview/) [LinkedIn](https://www.linkedin.com/company/storagereview-com) [YouTube](https://www.youtube.com/user/storagereview?sub_confirmation=1) [Email](mailto:info@storagereview.com) [Spotify](https://open.spotify.com/show/1y6VnznABhHeOSMOmbDTz0) [Reddit](https://www.reddit.com/r/StorageReview/) [Discord](https://discord.gg/TwMHb4azdC) [RSS Feed](https://www.storagereview.com/rss.xml) [TikTok](https://www.tiktok.com/@storagereview) [![StorageReview.com]()](https://www.storagereview.com) ≡ Menu - [Home](https://www.storagereview.com/) - [Storage Reviews](https://www.storagereview.com/review) - [End User Reviews](https://www.storagereview.com/consumer) - [Enterprise Reviews](https://www.storagereview.com/enterprise) - [Ubiquiti Reviews](https://www.storagereview.com/best/ubiquiti-reviews) - [SR Merch](https://store.storagereview.com) - [Leaderboards](https://www.storagereview.com/best) - [Best Storage Arrays](https://www.storagereview.com/best/storage-arrays) - [Best Enterprise SSDs](https://www.storagereview.com/best/enterprise-ssds) - [Best Servers](https://www.storagereview.com/best/servers) - [Best Enterprise GPUs](https://www.storagereview.com/best/enterprise-gpus) - [Best Desktops for Local AI](https://www.storagereview.com/best/desktops-local-ai) - [Best Laptops for Local AI](https://www.storagereview.com/best/laptops-local-ai) - [Best Local LLM Tools](https://www.storagereview.com/best/local-llm-tools) - [Agentic AI Hardware](https://www.storagereview.com/best/agentic-ai-hardware) - [Best SSDs & Hard Drives](https://www.storagereview.com/best_drives) - [Best Portable SSDs](https://www.storagereview.com/best/portable-ssds) - [Best Business Laptops](https://www.storagereview.com/best/business-laptops) - [Best Mobile Workstations](https://www.storagereview.com/best/mobile-workstations) - [Best Desktop Workstations](https://www.storagereview.com/best/desktop-workstations) - [Best Mini PCs](https://www.storagereview.com/best/mini-pcs) - [Best Laptop Battery Life](https://www.storagereview.com/best/laptop-battery-life) - [Storage Reference Guide](https://www.storagereview.com/storage-reference-guide) - [About SR](https://www.storagereview.com/about-storagereview) - [StorageReview.com Sweepstakes Rules and Regulations](https://www.storagereview.com/storagereview-com-sweepstakes-rules-and-regulations) Search [Home](https://www.storagereview.com/) » [News](https://www.storagereview.com/news) » Pliops and vLLM: Smarter KV Caching for LLM Inference # Pliops and vLLM: Smarter KV Caching for LLM Inference by Harold Fritts on March 14, 2025 [AI](https://www.storagereview.com/enterprise/ai)  ◇  [Enterprise](https://www.storagereview.com/enterprise) Pliops has announced a strategic partnership with the [vLLM Production Stack](https://github.com/vllm-project/production-stack), an open-source, cluster-wide reference implementation designed to optimize large language model (LLM) inference workloads. This partnership is pivotal as the AI community prepares to gather at the GTC 2025 conference. By combining Pliops’ advanced key-value (KV) storage backend with the robust architecture of the vLLM Production Stack, the collaboration sets a new benchmark for AI performance, efficiency, and scalability. ![]() Junchen Jiang, Head of the LMCache Lab at the University of Chicago, highlighted the collaboration’s potential, underscoring its ability to enhance LLM inference efficiency and performance. The joint solution brings advanced Vector Search and Retrieval capabilities by introducing a new petabyte-scale memory tier below High Bandwidth Memory (HBM). Computed KV caches are efficiently retained and retrieved using disaggregated smart storage, accelerating vLLM inference. For a primer on Pliops, check out our [deep dive article](https://www.storagereview.com/review/maximize-nvme-ssds-with-the-pliops-extreme-data-processor-xdp). ### About that KVCache Most large language models use transformer architectures, which rely on attention mechanisms involving Query, Key, and Value matrices. When generating tokens sequentially, transformers repeatedly compute attention, requiring the recalculation of previous Key and Value (KV) matrices, leading to increased computational cost. KV caching addresses this by storing previously computed KV matrices, allowing reuse in subsequent token predictions, significantly improving generation efficiency and throughput. However, this introduces new challenges. KV caches can become quite large, particularly during lengthy generations or batched inference with typical batch sizes of 32, eventually exceeding available memory. To address this limitation, a KV cache storage backend becomes essential. ### Pliops XDP LightningAI Deploying Pliops’ XDP LightningAI in data centers represents a paradigm shift in cost-effectiveness, providing substantial cost savings compared to traditional architectures. By adding dedicated XDP LightningAI servers alongside existing infrastructure, organizations can achieve remarkable savings, including a 67% optimization in rack space, a 66% reduction in power consumption, a 58% annual OpEx savings, and a 69% decrease in initial investment costs. Pliops continues to advance with its Extreme Data Processor (XDP), the XDP-PRO ASIC, complemented by a comprehensive AI software stack and distributed nodes. Utilizing a GPU-initiated Key-Value I/O interface, this solution enables unprecedented scalability and performance. Pliops’ XDP LightningAI delivers substantial end-to-end performance improvements, achieving up to 8X gains for vLLM inference, significantly accelerating Generative AI (GenAI) workloads. With the integration of cutting-edge industry trends such as DeepSeek, Pliops ensures robust adaptability for future AI developments. Pliops showcased these advancements at AI DevWorld, highlighting how XDP LightningAI revolutionizes LLM performance by significantly reducing computational power and cost. This demonstration illustrated Pliops’ commitment to enabling enterprise-scale sustainable AI innovation. ### Ongoing Collaboration Pliops positions organizations to maximize the potential of AI-driven insights and maintain a competitive advantage in a rapidly evolving technology landscape by providing instant access to actionable data and ensuring a seamless integration path. The future roadmap for the collaboration includes essential integration of Pliops’ KV-IO stack into the production stack, progressing towards advanced capabilities such as prompt caching across multi-turn conversations, scalable KV-cache offloading, and streamlined routing strategies. [Pliops](https://pliops.com) **Engage with StorageReview** [Newsletter](https://www.storagereview.com/storage_newsletter) |  [YouTube](https://www.youtube.com/user/StorageReview "Opens in a new window") | Podcast  [iTunes](https://podcasts.apple.com/gb/podcast/storagereview-com-storage-reviews/id1060681115 "Opens in a new window")/ [Spotify](https://open.spotify.com/show/1y6VnznABhHeOSMOmbDTz0 "Opens in a new window") |  [Instagram](https://www.instagram.com/storagereview/ "Opens in a new window") |  [Twitter](https://twitter.com/storagereview "Opens in a new window") |  [TikTok](https://www.tiktok.com/@storagereview? "Opens in a new window") |  [RSS Feed](https://www.storagereview.com/rss.xml) ![]() ### Harold Fritts I have been in the tech industry since IBM created Selectric. My background, though, is writing. So I decided to get out of the pre-sales biz and return to my roots, doing a bit of writing but still being involved in technology. Previous post: [KIOXIA’s 122TB LC9 SSD Targets AI, LLMs, and Hyperscale Storage](https://www.storagereview.com/news/kioxias-122tb-lc9-ssd-targets-ai-llms-and-hyperscale-storage) Next post: [CoolIT Unveils 4000W Cold Plate, Setting New Standard for Single-Phase Direct Liquid Cooling](https://www.storagereview.com/news/coolit-unveils-4000w-cold-plate-setting-new-standard-for-single-phase-direct-liquid-cooling) Trusted Vendors Products and solutions from our affiliate partners: - [ ![Ubiquiti Logo]() Ubiquiti ](https://store.ui.com/us/en?a_aid=StorageReview "Opens in a new window") - [ ![Newegg Logo]() Newegg ](https://click.linksynergy.com/fs-bin/click?id=g5terNMDhz0&offerid=1207190.18&subid=0&type=4 "Opens in a new window") - [ ![Amazon Logo]() Amazon ](https://amzn.to/4fLJUEW "Opens in a new window") Newsletter Subscribe to the StorageReview newsletter to stay up to date on the latest news and reviews. We promise no spam! 1 Leave this field empty if you’re human: ## Advertisement Content Categories [Facebook](https://www.facebook.com/@storagereview) [X](https://twitter.com/storagereview) [Instagram](https://www.instagram.com/storagereview/) [LinkedIn](https://www.linkedin.com/company/storagereview-com) [YouTube](https://www.youtube.com/user/storagereview?sub_confirmation=1) [Email](mailto:info@storagereview.com) [Spotify](https://open.spotify.com/show/1y6VnznABhHeOSMOmbDTz0) [Reddit](https://www.reddit.com/r/StorageReview/) [Discord](https://discord.gg/TwMHb4azdC) [RSS](https://www.storagereview.com/rss.xml) [TikTok](https://www.tiktok.com/@storagereview) Copyright © 1998-2025 Flying Pig Ventures, LLC Cincinnati, Ohio. All rights reserved. Manage your privacy To provide the best experiences, we and our partners use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us and our partners to process personal data such as browsing behavior or unique IDs on this site and show (non-) personalized ads. Not consenting or withdrawing consent, may adversely affect certain features and functions. Click below to consent to the above or make granular choices. Your choices will be applied to this site only. You can change your settings at any time, including withdrawing your consent, by using the toggles on the Cookie Policy, or by clicking on the manage consent button at the bottom of the screen. Functional Functional Always active The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network. Preferences Preferences The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user. Statistics Statistics The technical storage or access that is used exclusively for statistical purposes. The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you. Marketing Marketing The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes. Statistics Marketing Features Always active Special features Special purposes Always active - Manage options - Manage services - Manage {vendor\_count} vendors - [Read more about these purposes](https://cookiedatabase.org/tcf/purposes/) Accept Deny Manage options Save preferences Manage options - {title} - {title} - {title} Manage your privacy To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions. Functional Functional Always active The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network. Preferences Preferences The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user. Statistics Statistics The technical storage or access that is used exclusively for statistical purposes. The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you. Marketing Marketing The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes. Statistics Marketing Features Always active Special features Special purposes Always active - Manage options - Manage services - Manage {vendor\_count} vendors - [Read more about these purposes](https://cookiedatabase.org/tcf/purposes/) Accept Deny Manage options Save preferences Manage options - {title} - {title} - {title} Manage consent Manage consent ```json {"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.storagereview.com\/news\/pliops-and-vllm-smarter-kv-caching-for-llm-inference","url":"https:\/\/www.storagereview.com\/news\/pliops-and-vllm-smarter-kv-caching-for-llm-inference","name":"Pliops and vLLM: Smarter KV Caching for LLM Inference - StorageReview.com","isPartOf":{"@id":"https:\/\/www.storagereview.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.storagereview.com\/news\/pliops-and-vllm-smarter-kv-caching-for-llm-inference#primaryimage"},"image":{"@id":"https:\/\/www.storagereview.com\/news\/pliops-and-vllm-smarter-kv-caching-for-llm-inference#primaryimage"},"thumbnailUrl":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2022\/07\/StorageReview-Pliops-XDP-2.jpg","datePublished":"2025-03-14T14:01:57+00:00","dateModified":"2025-03-14T14:48:03+00:00","description":"Optimize LLM inference with Pliops and vLLM. Enhance performance, reduce costs, and scale AI workloads with KV cache acceleration.","breadcrumb":{"@id":"https:\/\/www.storagereview.com\/news\/pliops-and-vllm-smarter-kv-caching-for-llm-inference#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.storagereview.com\/news\/pliops-and-vllm-smarter-kv-caching-for-llm-inference"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.storagereview.com\/news\/pliops-and-vllm-smarter-kv-caching-for-llm-inference#primaryimage","url":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2022\/07\/StorageReview-Pliops-XDP-2.jpg","contentUrl":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2022\/07\/StorageReview-Pliops-XDP-2.jpg","width":1500,"height":1068},{"@type":"BreadcrumbList","@id":"https:\/\/www.storagereview.com\/news\/pliops-and-vllm-smarter-kv-caching-for-llm-inference#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.storagereview.com\/"},{"@type":"ListItem","position":2,"name":"News","item":"https:\/\/www.storagereview.com\/news"},{"@type":"ListItem","position":3,"name":"Pliops and vLLM: Smarter KV Caching for LLM Inference"}]},{"@type":"WebSite","@id":"https:\/\/www.storagereview.com\/#website","url":"https:\/\/www.storagereview.com\/","name":"StorageReview.com","description":"StorageReview.com is a leading provider of news and reviews throughout the entire IT stack - from the datacenter to the edge, and all points in between.","publisher":{"@id":"https:\/\/www.storagereview.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.storagereview.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.storagereview.com\/#organization","name":"StorageReview.com","url":"https:\/\/www.storagereview.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.storagereview.com\/#\/schema\/logo\/image\/","url":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2020\/02\/Storage-Reviews-2-2.png","contentUrl":"https:\/\/www.storagereview.com\/wp-content\/uploads\/2020\/02\/Storage-Reviews-2-2.png","width":344,"height":61,"caption":"StorageReview.com"},"image":{"@id":"https:\/\/www.storagereview.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/storagereview","http:\/\/youtube.com\/user\/storagereview"]}]} ```