Last updated: June 2026
Key Takeaways
- Open AI models improved faster than memory prices rose. In mid-2026, the cheapest upgrade for most people is a better small model, not more hardware.
- A 16GB computer now runs models that trade benchmark wins with systems that led public leaderboards a year ago, particularly on math and coding.
- "Open weights" does not mean home-runnable. A model's total parameter count sets your memory requirement; the active count only sets its speed.
Mid-2026 hands home users a strange split screen. Memory prices sit at multi-year highs because AI datacenters are buying DRAM faster than fabs can make it, and yet running capable AI on hardware you own has never been cheaper. The reason: open models shrank faster than the hardware inflated. Between early April and early June, three releases — Google's Gemma 4, Zyphra's ZAYA1-8B, and DeepSeek's V4-Flash — redrew the map of what runs on an ordinary computer, what needs a workstation, and what still belongs in a datacenter.
This guide sorts that map by the only variable that matters at home: how much memory you have.
Why Small Models Suddenly Got Good
Three forces converged over the past year, and each pushed in the same direction: more capability per gigabyte.
Mixture-of-Experts moved down-market. A Mixture-of-Experts (MoE) model stores many specialist subnetworks but activates only a few for each token it generates. Google's Gemma 4 26B A4B carries 26 billion parameters yet activates roughly 4 billion per token. Zyphra's ZAYA1-8B activates just 760 million of its 8.4 billion. The active count determines the computation per token, so these models generate text at a small model's speed while drawing on a larger one's knowledge — the mechanical reason a mini PC with no discrete GPU can now hold a conversation at reading speed.
Licensing caught up with the technology. Gemma 4 is the first Gemma released under the standard Apache 2.0 license rather than Google's previous custom terms. ZAYA1-8B ships under Apache 2.0 as well, and DeepSeek's V4 family uses MIT. For a home user the meaning is identical: no user thresholds, no terms that change mid-project, full permission to build on the weights commercially. The open half of the open-versus-closed divide got meaningfully more open this spring.
Test-time compute let small models punch up. Small models are now trained to reason in steps and spend more tokens on harder problems. Zyphra reports that ZAYA1-8B reaches 91.9 on the AIME 2025 math benchmark using its Markovian RSA sampling method — above DeepSeek-R1-0528, the leading open reasoning model of mid-2025, which is roughly eighty times larger. Vendor numbers deserve vendor-number skepticism, but the consistent third-party reading is that the strongest open models under 10 billion parameters now match or beat the previous year's leaders on competition math.
A fourth force runs underneath the others: memory-efficiency research keeps shrinking what inference requires, a trend we examined in our Google TurboQuant explainer. Every development compounds the same way: the floor keeps dropping while your hardware stays put.
What "Runs at Home" Actually Means
The single most useful rule of thumb: a model's quantized file size is your memory floor. If the download is 6.7GB, you need at least that much free memory before the operating system, the runtime, and the conversation take their share. Published figures are floors, not targets.
Quantization is what makes the math workable. Models are trained at 16-bit precision, but compressing the weights to 4-bit (Q4) cuts memory to roughly a quarter with a small, usually acceptable quality loss. Google goes further with Gemma 4 by publishing official quantization-aware-trained (QAT) checkpoints — models that learned to compensate for the precision loss during training and, per Google, perform nearly identically to full precision. Official Q4 GGUF files exist for every Gemma 4 size, ready for llama.cpp, LM Studio, and Ollama.
Now the costliest misconception in local AI: active parameters set a model's speed; total parameters set its memory bill. Google's own documentation spells this out for the 26B A4B — it activates 4 billion parameters per token, but all 26 billion must be loaded into memory to keep routing fast. The same logic scales up brutally. DeepSeek V4-Flash advertises 13 billion active parameters, which reads like laptop territory, but the model totals 284 billion, and a quantized checkpoint still lands around 160GB. If a spec sheet leads with the active count, find the total before you plan any hardware purchase.
Finally, speed. Generation between 10 and 20 tokens per second matches comfortable reading speed; below roughly 5, chat feels broken, though slow output is fine for overnight batch work.
The Three Releases That Redrew the Map
ZAYA1-8B: The Efficiency Outlier
Released May 6 by Palo Alto startup Zyphra under Apache 2.0, ZAYA1-8B is a Mixture-of-Experts reasoning model with 8.4 billion total parameters and 760 million active. It was trained end-to-end on 1,024 AMD Instinct MI300X GPUs with AMD networking — the first widely noted competitive model built with no Nvidia hardware in the loop. Training silicon has been a single-vendor bottleneck for years; every crack in that monoculture means more labs can afford to build open alternatives.
Zyphra reports 89.6 on the HMMT 2025 math benchmark, edging Claude Sonnet 4.5's 88.3, alongside the 91.9 AIME result above. Independent verification is still maturing, but reviewers agree it leads its size class on math and code by a wide margin.
The honest catch: ZAYA1-8B uses a custom architecture, and as of this writing it does not run on stock Ollama or llama.cpp. The supported path is Zyphra's forks of vLLM and Transformers; llama.cpp integration lives in an experimental development branch, and the community 4-bit builds (roughly 6 to 9GB — a fit for a 12GB GPU or a 16GB Mac) are unofficial. A tinkerer's pick today, a mainstream pick the week stock support merges — check the model page before downloading.
Gemma 4: Frontier Behavior From a Raspberry Pi to a Workstation
Google DeepMind released Gemma 4 on April 2, and the headline is not raw capability — it is the license: the first Gemma under Apache 2.0 after years of custom terms. The family spans five sizes: E2B and E4B for edge devices, a 12B Unified model for laptops added in a follow-up release, the 26B A4B Mixture-of-Experts, and a 31B dense model. All accept image input, the three smaller models handle audio natively, and context windows run 128K to 256K tokens. Google reported the 31B ranking third among open models on the Arena text leaderboard at launch, with the small models running fully offline on hardware as modest as phones and the Raspberry Pi.
Memory requirements, per Google's published table at Q4 precision: E2B needs 2.9GB, E4B 4.5GB, 12B 6.7GB, 26B A4B 14.4GB, and 31B 17.5GB — figures that already include about 20 percent loading overhead. Ecosystem support is the other half: day-one Ollama and LM Studio compatibility, the official QAT checkpoints above, and Ollama's latest release, which tested its Nvidia performance work on Gemma 4 26B and enables Vulkan by default for AMD and Intel GPUs. Getting started is one command:
ollama pull gemma4:e4b # 8GB machines
ollama pull gemma4:12b # 16GB machines
ollama pull gemma4:26b # 32GB machinesDeepSeek V4-Flash: The Reality Check
DeepSeek shipped its V4 family on April 24 under MIT with full open weights: V4-Pro at 1.6 trillion total parameters (49 billion active) and V4-Flash at 284 billion total (13 billion active), both with a one-million-token context window. Per Artificial Analysis, Flash lands within about five points of Pro on their Intelligence Index at a fraction of the running cost, placing it near MiniMax M2.7 on the intelligence-versus-size frontier. (M2.7 ships under a modified MIT license with commercial-use conditions worth reading before any deployment.)
The home reality is the one we flagged in our launch-day coverage: all 284 billion parameters must live in memory. Quantized builds target a single 80GB datacenter GPU or paired 48GB cards; on consumer gear, that translates to 128GB unified-memory Macs at the absolute borderline. Why include it in a home-hardware guide? Because of what it signals: frontier-adjacent quality at 13 billion active parameters, the strongest evidence yet that the active-parameter floor keeps dropping. Next year's Flash-class model may fit a 32GB box. Today's does not.
What Your Hardware Can Run in Mid-2026
Find your row below. One clarification first: a model must fit within whichever memory pool serves it — system RAM for CPU inference, VRAM for GPU inference. Apple Silicon pools the two, which is why unified memory stretches further than the same number on a PC spec sheet.
| Memory You Have | What Runs Well | Honest Expectation |
|---|---|---|
| 8GB RAM | Gemma 4 E2B or E4B; Phi-4-mini | Capable chat, summarization, and light coding help at reading speed on a modern CPU |
| 16GB RAM | Gemma 4 12B; Qwen3.6 mid-size models at Q4; ZAYA1-8B (experimental runtimes) | A genuine daily-driver assistant; the 26B A4B technically loads but with no headroom |
| 32GB RAM | Gemma 4 26B A4B or 31B; Qwen3.6-35B-A3B at Q4 | Near-frontier open quality; the current sweet spot for local AI |
| 12–24GB VRAM GPU | 12B-class models on 12GB; Gemma 4 26B A4B or 31B at Q4 on 24GB | The fastest responses and longest usable contexts per dollar |
| 64GB+ unified or multi-GPU | 70B-class dense models; large MoE experiments | Enthusiast territory; DeepSeek V4-Flash remains borderline even at 128GB |
Gemma figures are Google's published Q4_0 loading estimates, which include roughly 20 percent overhead. Context windows consume additional memory on top of these floors, and community quantizations vary by build.
Entry tier. A Raspberry Pi 5 with 8GB runs Gemma 4 E2B entirely offline — Google names the Pi as a supported target — the cheapest legitimate way into a dedicated household AI endpoint.
Check Price on Amazon: CanaKit Raspberry Pi 5 Kit
16GB tier. This is most laptops sold since 2023, and the right first move costs nothing: install Ollama or LM Studio and pull Gemma 4 12B. At 6.7GB loaded, it leaves real headroom — the biggest capability jump per dollar in this guide, because the dollar amount is zero.
32GB tier. This is where local AI stops feeling like a compromise. The 26B A4B and 31B both run comfortably, and a 32GB mini PC makes an excellent always-on AI server for the whole household — our mini PC guide for local AI covers setup and network isolation. The Beelink SER9 PRO+ with 32GB of LPDDR5X is a strong fit here.
Check Price on Amazon: Beelink SER9 PRO+ (32GB)
GPU tier. If you already own a desktop, a used 24GB card remains the best speed-per-dollar play, and the renewed RTX 3090 is still the pick in our full local AI hardware guide. It holds Gemma 4 31B at Q4 entirely in VRAM with room for a long context.
Check Price on Amazon: RTX 3090 24GB (Renewed)
Beyond. At 64GB of unified memory and up, you reach 70B-class dense models and serious MoE experimentation. Genuinely capable territory — but at current memory prices, climb only when a model you specifically want demands it.
What Still Doesn't Run at Home
Honesty requires the other half of the map. DeepSeek V4-Pro (1.6 trillion parameters), GLM-5.1 (744 billion), Kimi K2.6 (trillion-class), and Llama 4 Maverick all publish weights you can download for free — and none runs on anything a normal household owns. Llama 4 adds a wrinkle worth knowing: Meta's license excludes EU-domiciled users and requires a separate license above 700 million monthly active users, exactly the conditions the Apache 2.0 and MIT class avoids. Downloading is free; the silicon and the electricity are not. Our April open-model class breakdown sorts the full lineup into consumer-runnable, borderline, and datacenter-only tiers.
The closed frontier still leads, too. The hardest long-horizon and agentic work remains the territory of hosted models, which charge for the privilege in both dollars and data — our Claude Fable 5 coverage lays out that trade in 2026. But the trend line matters more than today's gap. A year ago, local AI meant noticeably worse AI; in mid-2026, it means slightly behind the frontier for most everyday work — drafting, summarizing, document chat, routine coding — on hardware whose terms can never change underneath you. The floor rises every quarter. That is the whole story of the small-model surge.
Frequently Asked Questions
What is the best small AI model to run at home in 2026?
For most people, Gemma 4: the 12B on a 16GB machine, or the 26B A4B on a 32GB machine. It pairs a permissive Apache 2.0 license with official quality-preserving quantizations and support in every major runtime. If math and code on a small footprint is the priority, ZAYA1-8B leads its class, runtime caveats permitting. For agent workflows and multilingual work, the Qwen3.6 family is the strongest alternative.
How much RAM do I need to run AI models locally?
8GB is the floor, 16GB is comfortable, and 32GB is the current sweet spot. The working rule: take the model's quantized file size and add at least a third for the operating system, the runtime, and your conversation context. A 6.7GB model like Gemma 4 12B is happy on 16GB; the 14.4GB 26B A4B wants 24GB or more to breathe.
Can I run DeepSeek V4 at home?
V4-Pro, no — it is datacenter hardware by any definition. V4-Flash, only at the extreme edge: 128GB unified-memory Macs or multi-GPU rigs running aggressive quantization. Hosted access exists, but your prompts then travel through DeepSeek's servers — defeating the purpose for privacy-minded users. For most readers, the practical answer is to run the strongest model your memory tier supports from the table above.
Do I need a GPU to run AI models locally?
Not anymore for the small and MoE class: recent CPUs and Apple Silicon run models like Gemma 4 12B and 26B A4B at conversational speed. A dedicated GPU still earns its cost for 30B-plus models at speed, long contexts, and heavy batch work.
What is the difference between total and active parameters?
Total parameters are everything the model stores; active parameters are the subset used to generate each token. Active count determines speed. Total count determines memory, because the entire model must be loaded for the routing to work. When evaluating any MoE model, find the total parameter count first — it is the number your hardware actually pays for.
Should I wait for cheaper RAM before trying local AI?
No — start with the memory you have. Models are improving faster than memory is getting cheaper, which means your current machine runs better AI every quarter without a single hardware change. Upgrade when a specific model you want demands it, not in anticipation.
