Updated August 16, 2026: Initial publication. Model sizes and memory figures verified against official model releases as of this date; this table is refreshed quarterly because model sizes churn.
Every hardware guide for local AI eventually comes down to one question: will the model fit? This page answers it with numbers, then goes one step further than the spec-sheet math. Where a model class appears in the tables below, we have run it in the StorageReview lab, and the hardware recommendations link to systems we have actually tested on our desktop and laptop Local AI leaderboards.
The short version: parameter count alone no longer predicts hardware requirements. Quantization shrinks weights by roughly 4x, mixture-of-experts models run far lighter than their total parameter counts suggest, and agentic workloads add a memory tax that most sizing guides skip entirely. Here is how to budget all three.
Model Memory Requirements
File sizes below are the common Q4_K_M quantized releases (or the native format where noted) as published on Ollama and Hugging Face in August 2026. The “Memory to Run” column adds working overhead and a moderate context window; figures marked with an asterisk are computed estimates, while the GPT-OSS and DeepSeek 671B figures are the vendors’ own statements.
| Model | Parameters | Q4 / Native Download | Memory to Run* | Runs On |
|---|---|---|---|---|
| Llama 3.1 8B | 8B | 4.9 GB | ~6 to 8 GB | 8GB VRAM laptops and up |
| Phi-4 14B | 14.7B | 9.1 GB | ~11 to 12 GB | 16GB VRAM or 64GB shared-memory laptops |
| GPT-OSS 20B (MXFP4) | 21B (3.6B active) | 12 to 14 GB | ~13 to 16 GB | 16GB VRAM, shared-memory laptops |
| Mistral Small 3.2 24B | 24B | 15 GB | ~18 to 20 GB | 24GB RTX PRO laptops |
| Gemma 3 27B | 27B | 17 GB | ~20 to 22 GB | 24GB RTX PRO laptops, unified-memory desktops |
| Qwen3 30B-A3B | 30.5B (3.3B active) | 19 GB | ~22 to 24 GB | 24GB VRAM (tight) or unified memory |
| QwQ 32B / DeepSeek-R1 32B | 32B | 20 GB | ~22 to 24 GB | 24GB VRAM (tight) or unified memory |
| Llama 3.3 70B / R1 Distill 70B | 70B | 43 GB | ~48 to 50 GB | 96GB+ unified memory, RTX PRO 6000 towers |
| GPT-OSS 120B (MXFP4) | 117B (5.1B active) | 65 GB | ~65 to 80 GB | 96GB to 128GB unified memory, 96GB GPUs |
| DeepSeek-R1 671B (full) | 671B (37B active) | 404 GB | 450+ GB | Out of reach for desktops; rack-scale territory |
*Estimates assume roughly 8K of active context and about 15 percent runtime overhead on top of the weight file. Larger contexts cost substantially more; see the agentic tax below.
Two patterns worth noticing. First, mixture-of-experts changes the math: GPT-OSS 120B carries 117 billion parameters but activates only 5.1 billion per token, so it runs on a single 96GB to 128GB memory pool, and OpenAI states the target as a single 80GB GPU. We have run it on the HP Z2 Mini G1a, a 1-liter machine with no discrete GPU at all. Second, the full DeepSeek-R1 at 671B parameters is in a different universe: 404GB just to download at Q4, roughly 450GB of combined memory to run it acceptably. Nothing on any of our leaderboards runs it, and no desktop should try.
What Runs Where
8GB VRAM laptops (RTX PRO 500 to 2000 class). Comfortable with 7B to 8B models at Q4 and the smaller Phi and Gemma variants. The ceiling is real: in our testing, a 13B model that wanted about 12GB of graphics memory simply did not finish on 8GB cards. Our Best Laptops for Local AI page covers the field.
24GB VRAM systems (RTX PRO 5000 laptops, desktop RTX cards). The sweet spot for 14B to 27B models with context to spare, and the fastest way to run them; our Dell Pro Max 18 Plus leads that field at 185 tokens per second on Phi. The 32B class fits at Q4 but leaves little room for context, which matters more than you think once agents are involved.
Unified and shared memory systems (96GB to 128GB). AMD Strix Halo machines can assign up to 96GB of system RAM to the GPU, and GB10 systems like the NVIDIA DGX Spark carry 128GB of coherent memory. This class runs 70B models at Q4 and GPT-OSS 120B, trading speed for capacity. The Best Desktops for Local AI leaderboard ranks them, and the same architecture reaches laptops in the HP ZBook Ultra G1a 14, which loaded DeepSeek-R1 70B in our testing.
RTX PRO 6000 workstation towers (96GB per card, up to 384GB tested). Speed and capacity at once: 70B models at higher quantization with full context, GPT-OSS 120B with headroom, or several models resident simultaneously for multi-agent pipelines. Our Best Desktop Workstations leaderboard covers the class, topped by the four-GPU-capable HP Z8 Fury G6i.
The Agentic Tax: Context Is a Second Memory Budget
Sizing from the weight file alone works for short chat sessions. Agents break that assumption. An agent loops: it reads tool output, files, and prior steps back into its context window, and every token held in context costs memory in the KV cache on top of the weights. The costs are not small.
| Model Class | KV Cache at 32K Context | KV Cache at 128K Context | Rough Cost per 1,000 Tokens |
|---|---|---|---|
| 8B (Llama 3.1 8B, FP16 KV) | 4.19 GB | 16.78 GB | ~0.13 GB |
| 70B (Llama 3.3 70B, FP16 KV) | 10.49 GB | 41.94 GB | ~0.33 GB |
Read that table against the sizing one above and the problem is obvious. A 70B model at Q4 is 43GB of weights, but at a full 128K context it needs roughly 85GB in total, which is double the weights. An 8B model at 128K spends more memory on context (16.78GB) than on its own Q4 weights (4.9GB). KV cache quantization helps, with FP8 halving those figures and INT4 quartering them at some quality cost, but the planning rule stands: for agentic work, budget context like a second model.
Serving multiplies the tax again. If a system hosts several agents or users concurrently, each request holds its own KV cache. vLLM, the serving stack we benchmark with on our Local AI leaderboards, pre-allocates 90 percent of GPU memory by default precisely because KV space is what determines how many concurrent requests survive without throughput-killing preemption. A machine that runs one chat comfortably can be undersized for three agents.
System RAM: The Forgotten Third Leg
For GPU-only inference, system RAM just needs to stay out of the way, and 32GB to 64GB is a comfortable floor on the machines we test. It becomes decisive in two cases. Offloading, where layers that do not fit in VRAM spill to CPU, works through memory-mapped model files, so system RAM should at least match the model file size plus OS headroom. And on unified memory machines, system RAM is the GPU memory, which is exactly why 64GB and 128GB configurations dominate our Local AI leaderboards. Ollama’s own floor guidance runs 8GB of RAM for 7B models, 16GB for 13B, and 32GB for the 33B class; treat those as minimums, not targets.
Storage: The Leg Everyone Skips
Model libraries get large quickly. The ten models in our sizing table total roughly 240GB in their common quantizations, and a working library with a few variants, embedding models for retrieval, and vector stores for agent memory can pass 500GB before any project data arrives. On top of that, agentic workflows generate real scratch traffic: logs, checkpoints, and retrieval indexes that live alongside the models.
Load time is where drive class shows up. The arithmetic is simple: a 65GB GPT-OSS 120B file reads in about 109 seconds at SATA speeds, roughly 17 seconds on a Gen3 NVMe drive, and under 9 seconds at Gen4 rates, first load only, since the OS page cache makes reloads near-instant. Real-world loaders do not always sustain full drive speed, so we plan to publish our own measured model-load numbers across drive classes; watch the changelog. Our Storage Leaderboard covers the drives themselves.
Practical guidance: 2TB NVMe is the sensible floor for a dedicated AI workstation, 4TB if the machine hosts agents that accumulate state, and Gen4 or better if you swap between large models regularly.
Agentic AI Hardware FAQ
How much VRAM do I need to run a 70B model locally?
About 48 to 50GB at Q4 with moderate context, which is why this class belongs to 96GB unified memory machines and 96GB workstation GPUs rather than any consumer card. Push the context toward 128K and the total approaches 85GB. Run it at Q8 and the weights alone are 75GB.
Can I run the full DeepSeek-R1 locally?
Not on anything this site ranks. The full 671B model is a 404GB download at Q4 and wants roughly 450GB of combined memory to run acceptably. The practical local path is the official distills, and the 32B and 70B distills in our sizing table capture much of the capability at desktop-class requirements.
Do mixture-of-experts models change the hardware math?
Substantially. MoE models activate a fraction of their parameters per token, so memory scales with total parameters but speed scales closer to active parameters. GPT-OSS 120B (5.1B active) and Qwen3 30B-A3B (3.3B active) both run faster than their sizes suggest on the memory pools that fit them. You still need the memory for all the weights; you just get more speed per gigabyte.
How much storage does an AI workstation need?
Plan 2TB NVMe minimum, 4TB for agent hosts. A modest model library alone runs hundreds of gigabytes, agents accumulate logs and vector stores, and the largest single files (65GB and up) make slow drives painful every time you load or swap a model.
What hardware should I buy for local agents specifically?
Prioritize memory over raw speed. Agents hold long contexts and sometimes run concurrently, so the KV cache budget matters as much as the weight budget. A 96GB to 128GB unified memory system on our desktop Local AI leaderboard is the most economical entry, and RTX PRO 6000 towers on the workstation leaderboard are the answer when speed and concurrency both matter.
On the Horizon: GB300 Is in the Lab
Everything above reflects hardware we have tested. One system that will change the math is already on the bench: we have an NVIDIA GB300 Grace Blackwell Ultra deskside system in for review.
The reason it matters for this guide is memory. Every recommendation on this page works around a hard ceiling, splitting models across GPUs, leaning on quantization, or accepting CPU offload when weights and KV cache will not fit. NVIDIA specs the GB300 deskside systems with several hundred gigabytes of coherent memory addressable by the accelerator, which is a different order of magnitude from the 96GB-class cards that anchor the tables above. Model classes that currently require a multi-GPU tower, or that we list as impractical to run locally at all, move into single-box range.
We are not going to put numbers to that until we have run it. No figure on this page is based on GB300, and nothing here has been adjusted in anticipation of it. When the review publishes, this guide gets revised against measured results and the change will be noted in the log below.
How We Maintain This Guide
Model sizes and the popular-model roster change quarterly, faster than any hardware cycle. We re-verify the sizing table against official releases on a quarterly pass, and lab receipts are added as new systems clear our Local AI testing. Changes land in the dated note at the top of the page. Sizing figures marked as estimates use the published weight file plus measured KV cache costs and standard runtime overhead; vendor-stated figures are identified as such.
Amazon