Best Open-Source Local LLMs by Hardware Tier (July 2026)

Last updated: July 2026

The open-source AI floor keeps dropping while your hardware stays put. The gap between the strongest open weights and the best closed models has narrowed to single digits on the coding and agent benchmarks most people care about. This guide ranks the open-weight models worth running in mid-2026 by the only spec that decides what you can run: memory. We split them into four hardware tiers, compare the models inside each, and chart the top tier against the closed frontier. The practical question first: which open model fits the hardware you have? The bigger reason it matters comes at the end.

How to read this guide: memory is the spec that matters

The most important specification for local inference is memory, not CPU or GPU teraflops. A model's weights must fit entirely in RAM or VRAM; if they do not, the system swaps to disk and throughput collapses from 30-plus tokens per second to 3 to 5. Quantization makes the math workable: compressing weights to 4-bit (Q4) cuts memory to roughly a quarter with a small, usually acceptable quality loss. A model's quantized file size is your memory floor, plus headroom for the OS, runtime, and context window. The four tiers are defined by total addressable memory: unified RAM on Apple Silicon and AMD APUs, or VRAM on a discrete GPU.

Tier 1: Entry (8 to 16 GB, no GPU required)

This is most laptops sold since 2023 and any modern mini PC. The headline pick is Gemma 4 12B Unified (June 3, 2026): a dense model handling text, images, audio, and video in one encoder-free architecture, with a 256K context. At Q4 it needs roughly 7 GB, and it shipped with day-one support in Ollama, LM Studio, and llama.cpp. The Gemma 4 edge variants run on a Raspberry Pi 5 or a phone.

Memory figures are Q4 floors including about 20 percent loading overhead; leave headroom for your OS and context.

For a dedicated always-on box, a Raspberry Pi 5 runs the edge models comfortably and doubles as a home network device.

Check Price on Amazon: CanaKit Raspberry Pi 5 Starter Kit PRO

See our guide to what AI models you can actually run at home.

Tier 2: Enthusiast (24 to 32 GB; one 24 GB GPU or a 32 GB Mac/APU)

This is where local AI stops feeling like a compromise. The default for most readers is Gemma 4 26B MoE: Apache 2.0, multimodal, and runnable in about 16 GB at Q4 because it activates only 3.8 billion parameters per token. For a higher ceiling on agentic coding and long context, Qwen3.6-35B-A3B is the stronger choice; its Q4 build weighs 20.9 GB and runs on a 32 GB Mac or 24 GB GPU.

Two clean hardware paths: a Mac Mini with 32 GB of unified memory handles these at usable speed, and a used RTX 3090 with 24 GB of VRAM remains the best dollar-per-VRAM value in 2026, holding Gemma 4 31B Dense at Q4 with room for a long context.

Check Price on Amazon: NVIDIA RTX 3090 24 GB (Renewed)

Check Price on Amazon: MINISFORUM UM880 Plus (32 GB / 1 TB)

For GPU-versus-mini-PC trade-offs, see our local AI hardware guide and mini PC roundup.

Tier 3: Prosumer borderline (64 to 128 GB unified memory)

Two large mixture-of-experts models are too big for a typical home machine but can be coaxed onto a 128 GB Mac Studio with aggressive quantization, putting frontier-adjacent open weights in a single box. The honest catch: throughput is modest and the tooling is rough.

MiniMax M3 ships under the MiniMax Community License, not Apache 2.0 or MIT. Its predecessor restricted commercial use without written authorization, so confirm the terms before building on it.

On paper, DeepSeek V4-Flash is the release local builders spent two years asking for: open weights under MIT, near-frontier results, a download price of zero. In practice the realistic entry point is 96 GB of VRAM or 128 GB of unified memory, and every consumer run today depends on experimental forks. Below that, run a model that fits well. Our DeepSeek V4-Flash reality check and Ryzen AI Max+ 395 reality check cover the details, including the prefill speeds the demos skip.

Check Price on Amazon: MINISFORUM X1 Pro 370 (64 GB / 1 TB)

Tier 4: Workstation and data-center class (192 GB and up)

These are the open models that genuinely rival the closed frontier. All have downloadable weights, and almost none run at home; they need multi-GPU servers or 256 GB-plus unified-memory machines. For nearly every individual these are API models that ship their weights, which matters the day you want to own the stack.

How the best open weights compare to the closed frontier

The table charts the top open models against the closed frontier on SWE-bench Pro, the least saturated coding benchmark in wide use (real, recent GitHub issues). Claude Fable 5 is included as a reference, the strongest published score, though it is currently unavailable (see below).

Closed-frontier figures are from vendor system cards. Open-model scores marked "vendor" are self-reported or early numbers not yet independently replicated; Z.ai published no benchmarks for GLM-5.2 at launch. Read the ordering, not the decimals.

Two models report their headline coding result on SWE-bench Verified rather than Pro: DeepSeek V4-Pro lands around 80 percent there on vendor numbers, inside the frontier pack, and Nemotron 3 Ultra reports roughly 72 percent at peak while leading the group on long-context retrieval and throughput. The honest counterweight is reasoning: on GPQA Diamond the closed frontier sits at 93 to 94 percent while the best open figure (GLM-5.2, vendor) is around 80 percent, and DeepSeek's own paper puts its V4 generation three to six months behind the frontier on the hardest reasoning. The takeaway is not that open weights have caught the frontier outright; it is that on the agentic coding most teams run daily, the deciding factor is increasingly hardware and licensing, not capability. See our DeepSeek V4 release coverage for the trade-offs behind the largest models.

Privacy and security: local is not automatic

Running a model on your own hardware is the necessary first step for digital sovereignty, not the sufficient one. A local stack that binds to 0.0.0.0 on your main LAN with no isolation is not meaningfully more private than a cloud service, because anything on your network can reach it.

The baseline hygiene: keep Ollama on its default localhost binding, run inference in a Docker container, and put AI workloads on their own VLAN if your router supports it. Monitor outbound DNS with Pi-hole, and pull weights only from verified sources. For larger builds like a personal knowledge base, see our guide to a private knowledge base with Obsidian and Ollama.

So which one should you pick?

For most readers, Gemma 4 26B MoE: Apache 2.0, multimodal, runs in 16 GB at Q4. With a 24 GB GPU or 32 GB Mac, step up to Qwen3.6-35B-A3B for the higher coding and long-context ceiling. On a 128 GB Mac with frontier-class ambitions, DeepSeek V4-Flash is reachable if you are willing to compile forks. If you are fine with hosted access, DeepSeek V4-Pro or GLM-5.2 give frontier-adjacent results at a fraction of closed-model token cost, with MIT weights you can self-host later. One caution before you ship anything commercial: MiniMax M3 (custom license), Kimi K2.7 (Modified MIT with a monthly-active-user cap), and Meta's Llama 4 (EU restriction plus a 700-million-MAU clause) all carry strings; Apache 2.0 and MIT models are the clean choices.

The bigger picture: when the frontier can be switched off

For most of 2026 the case for local AI was privacy, cost, and offline reliability. In June it gained a sharper edge. Anthropic's two most capable models, Fable 5 and its larger sibling Mythos 5, sit above the Opus line. On June 12, 2026, a US Commerce Department export-control directive ordered access blocked for any foreign national, a category that includes green-card holders, visa workers, and a company's own non-citizen employees. With no way to verify citizenship in real time at the API layer, Anthropic disabled both models for everyone to comply. Every other Claude model, including Opus 4.8, kept running.

This is policy, not a product retirement, and Anthropic says it disagrees and is working to restore access. But the lesson stands regardless: the most capable model in the world can go uncallable overnight for reasons that have nothing to do with your work. We track it in our coverage of what Fable 5 and Mythos 5 are and whether Fable 5 is coming back.

That is the throughline of everything we cover: whoever controls the infrastructure controls the experience. A capable open model on hardware you own never changes its terms mid-project, never hits a usage cliff, and never goes dark over a policy you had no part in. The frontier gap is real and we will keep being straight about it, but for most everyday work an open model you run yourself is already enough, and it stays yours. To start, see our picks by exact VRAM and full hardware guide.

Frequently Asked Questions

What is the difference between open source and open weights?

Open weights means the model parameters are downloadable, so you can run and fine-tune the model yourself. Open source, strictly, also releases the training code and data under a permissive license. Most models here are open weights; the Apache 2.0 and MIT ones (Gemma 4, Qwen3.6, GLM-5.2, DeepSeek V4) come closest to full open source, and NVIDIA's Nemotron 3 Ultra publishes its training data and recipes too.

Can I run Qwen 3.7 Max or another flagship frontier model locally?

No. Qwen 3.7 Max is proprietary and API-only, with no downloadable weights, despite Alibaba's open-weight history. The same goes for GPT-5.5, Claude Opus 4.8, and Gemini 3.1 Pro. If you want a self-hostable Qwen, the current open option is Qwen3.6-35B-A3B (Apache 2.0). The frontier flagships appear here only as comparison points.

Do I need a GPU to run a local LLM in 2026?

No. Apple Silicon Macs with 16 GB or more run quantized small models at conversational speed, and with 32 GB they handle Qwen3.6-35B-A3B and Gemma 4 31B at Q4. Modern AMD mini PCs with 32 GB of DDR5 run Gemma 4 26B MoE with no discrete GPU. A GPU still helps for sustained high-throughput inference, but it is no longer required to start.

How much memory do I need for a 30B-class model at Q4?

Plan for roughly 20 GB of RAM or VRAM for the weights, plus 4 to 8 GB of headroom for the OS, context, and runtime. A 32 GB machine is the comfortable minimum; 64 GB gives real room for multiple models or long contexts. The borderline models (DeepSeek V4-Flash, MiniMax M3) need a 128 GB unified-memory machine.

Is the strongest open coding model really close to the frontier?

On real-world coding the gap is now small. On SWE-bench Pro, GLM-5.2 and MiniMax M3 post vendor-reported scores ahead of GPT-5.5 and Gemini 3.1 Pro, with Opus 4.8 still on top. On the hardest science reasoning (GPQA Diamond) the frontier still leads clearly. Treat vendor numbers with care until independent evaluators replicate them, but the direction is consistent.

Does the Fable 5 and Mythos 5 suspension change these recommendations?

For local inference, no: Gemma 4 26B MoE and Qwen3.6-35B-A3B are still the picks for most hardware. What it changes is the argument. It is a concrete example of frontier access being revoked by policy, with no warning and no recourse for ordinary users, while models you run yourself were unaffected because there was nothing to switch off.