Last updated: September 2026
Key Takeaways
- Yes, it runs at home: a 5.95 GB or 7.21 GB download under Apache 2.0. PrismML measures 91 to 97 tokens per second on an RTX 4090 and about 28 on an M5 Pro laptop.
- It does not run in Ollama, stock llama.cpp, or ordinary MLX loaders today. You need PrismML's fork or its bundled MLX loader, and two failure modes produce wrong output with no error.
- The 98.2% quality figure is a 20-benchmark average at maximum reasoning effort. It is 96.0% at medium effort and about 75% on two agentic coding benchmarks reported outside the average. All numbers are vendor-reported.
Yes. Bonsai 2 27B, released by PrismML on September 17, 2026, packs Alibaba's Qwen3.8-27B into a 5.95 GB file, small enough for an 8GB GPU or a 16GB laptop, under an Apache 2.0 license. It will not load in Ollama or stock llama.cpp yet, and the 98.2% quality claim needs three qualifiers.
We checked the model cards, the whitepaper, PrismML's runtime documentation, and the llama.cpp issue tracker on launch day. Every quality and speed figure below is PrismML's own unless we say otherwise.
What Bonsai 2 27B Is
Bonsai 2 27B is PrismML's ternary rebuild of Qwen3.8-27B: the same 27-billion-parameter architecture, with nearly every language weight stored as one of three values instead of a 16-bit number.
| Item | Specification |
|---|---|
| Base model | Qwen3.8-27B, architecture unchanged |
| Weight format | Ternary (-1, 0, +1), one FP16 scale per 128 weights, Hadamard-rotated; 1.72 bits per weight |
| GGUF packs | PTQ1_0 at 5.95 GB, PQ2_0 at 7.21 GB, language model only |
| Vision tower | Optional 0.63 GB file |
| MLX pack | 8.60 GB, vision tower included |
| Context window | 262,144 tokens |
| License | Apache 2.0 |
| FP16 equivalent | About 54 GB |
Sources: PrismML's GGUF and MLX model cards, September 17, 2026.
Size matters because text generation at home is limited by how fast weights move from memory to the processor, and a file one ninth the size moves one ninth the data per token. PrismML counts a 4-bit build of the same model at 17.6 GB, too large for a 16GB machine.
New in this generation is a rotation: PrismML applies a fixed Hadamard transform to each weight matrix before reducing it to three values, and the runtime must apply the matching transform on every step. That detail is what breaks compatibility.
PrismML does not publish how the ternary weights are produced. The press release says the model was trained using Google v5 TPUs; the whitepaper covers the format, kernels, and evaluation, with no training section. The weights are open. The recipe is not.
What Hardware Runs Bonsai 2 27B
An 8GB GPU or a 16GB laptop is the realistic floor, and the smaller PTQ1_0 pack is the one that fits there.
Memory: Weights Are Not the Whole Bill
The 5.95 GB figure is weights only. A running model also needs a context cache and runtime buffers, and PrismML has published no peak-memory table for Bonsai 2. The closest evidence is the first Bonsai 27B, which shares the architecture. An independent benchmark on an RTX 5060 Ti measured 7.4 GiB of VRAM for that model's 7.17 GB file at a 4,000-token context, and the full 262K context in 12.9 GiB with a 4-bit cache.
| Your hardware | Pack to use | What to expect |
|---|---|---|
| 8GB GPU | PTQ1_0 (5.95 GB) | Short context, text only. PQ2_0 leaves no room once buffers are counted. |
| 12GB to 16GB GPU | Either GGUF pack | Comfortable. On 16GB, the full 262K context is plausible with a 4-bit cache. |
| 16GB Apple Silicon Mac | MLX pack (8.60 GB) | Runs with limited context. PrismML's first-generation guidance was 16GB minimum, 24GB for long context. |
The GPU rows are our estimates from first-generation measurements, not tests of Bonsai 2. CPU-only inference is supported by PrismML's fork, with no published speeds.
Measured Speeds
PrismML's decode speeds run from about 28 tokens per second on an M5 Pro laptop to between 130 and 143 on an RTX 5090. Its model card and whitepaper disagree by up to 10 percent on the same hardware, so the table shows both.
| Hardware | Model card (tok/s) | Whitepaper (tok/s) |
|---|---|---|
| RTX 5090 (32GB) | 129.9 | 142.5 |
| RTX 4090 (24GB) | 91.1 | 96.7 |
| Apple M5 Max laptop | 47.0 | 46.8 |
| Apple M5 Pro laptop | 28.1 | 27.7 |
| Apple M4 Pro laptop | 18.0 | 18.0 |
Decode speed per PrismML, faster pack shown (PTQ1_0 on the RTX 4090, PQ2_0 elsewhere). The M4 Pro figures and the model card's M5 Max figure come from an earlier build. No consumer GPU below the RTX 4090 appears in PrismML's tables.
PrismML has published nothing for older consumer cards such as the RTX 3060 or 3090. For scale, the independent first-generation test measured 44.4 tokens per second on a 16GB RTX 5060 Ti.
Tokens per second is not time to an answer. Bonsai 2 reasons at its highest effort setting by default, and PrismML's benchmark runs allowed up to 81,920 output tokens on the hardest tasks. At 28 tokens per second, that ceiling is about 49 minutes for one response. Laptop users will want the medium setting, and Catch 2 shows what that costs.
Catch 1: It Does Not Run in Ollama or Stock llama.cpp Yet
As of September 17, 2026, no mainstream one-click runtime runs Bonsai 2 27B: not Ollama, not a stock llama.cpp build, and not an ordinary MLX loader.
The cause is the rotation. Stock runtimes do not apply the Hadamard transform, and PrismML's two new tensor types, PQ2_0 and PTQ1_0, do not exist upstream. PrismML's format documentation says so directly, and a llama.cpp feature request opened within hours of launch asks upstream to add the types or reject them with a clear error.
| Runtime | Status | What happens |
|---|---|---|
| Ollama (current releases) | Does not run | Import fails. One user on Ollama 0.33.2 reported: tensor "output.weight" size overflow |
| Stock llama.cpp and apps that bundle it | Does not run | Both packs are rejected as unknown types. PrismML warns that a Q2_0 file loads silently and outputs gibberish. |
| LM Studio | Not documented | PrismML's guide covers the first-generation MLX model only. |
| Ordinary MLX loaders such as mlx_lm | Wrong output | PrismML's MLX card says they skip the transform and return wrong output, not an error. |
| PrismML's fork (prism-b10658 or newer) or its bundled MLX loader | Works | CUDA, Metal, and CPU through the fork; Apple Silicon through the MLX pack's own loader. |
Verified September 17, 2026 against PrismML's documentation, both model cards, and llama.cpp issue 29058. The Ollama error is a third-party report.
One more trap sits on the model page. Hugging Face auto-generates "Use this model" commands for Ollama, LM Studio, and mlx_lm above PrismML's warnings, and the llama.cpp and Ollama versions point at the F16 file, a 53.8 GB full-precision copy. Ignore them.
The Route That Works Today
PrismML's Bonsai-demo repository is the supported path, and the one PrismML calls the source of truth when its pages disagree. Its setup script fetches the right pack and a matching binary.
- Clone the repository and run setup. PrismML's documentation says Bonsai 2 27B is the default model; confirm that in the README.
- Start the server for an OpenAI-compatible API and chat interface on port 8080. Or run a fork binary, prism-b10658 or newer from PrismML's llama.cpp releases, against a downloaded pack.
- On Apple Silicon, use the loader in the MLX pack's runtime folder. Do not point mlx_lm at it.
git clone https://github.com/PrismML-Eng/Bonsai-demo.git
cd Bonsai-demo
./setup.sh
./scripts/start_llama_server.sh
# or, with a fork binary already extracted to ./bin:
hf download prism-ml/Ternary-Bonsai-2-27B-gguf Ternary-Bonsai-2-27B-PTQ1_0.gguf --local-dir .
./bin/llama-cli -m Ternary-Bonsai-2-27B-PTQ1_0.gguf -ngl 99 -fa on -c 8192 --temp 1.0 --top-p 0.95 --top-k 20Two cautions. This route runs binaries from a vendor's fork: the source is public, so build it yourself if your threat model calls for that, and do not mix the fork's libraries with a stock llama.cpp install. The server is also an unauthenticated API. Keep it bound to localhost, or isolate the machine as described in our mini PC guide for local AI.
Catch 2: The 98.2% Is an Average, and Agent Work Is Where It Thins
The 98.2% figure is real but narrow: it averages 20 benchmarks at the highest reasoning effort, and the results PrismML reports outside that average are much lower.
| Measure | Qwen3.8-27B FP16 | Bonsai 2 27B | Retained |
|---|---|---|---|
| 20-benchmark average, xhigh effort | 85.4 | 83.9 | 98.2% |
| 20-benchmark average, medium effort | 82.6 | 79.3 | 96.0% |
| Math (4 benchmarks) | 97.06 | 96.57 | 99.5% |
| Coding (4 benchmarks) | 82.17 | 81.58 | 99.3% |
| Agentic and tool calling (2) | 79.74 | 77.57 | 97.3% |
| Vision (5) | 81.64 | 78.59 | 96.3% |
| Terminal-Bench 2.1 (outside the average) | 69.7 | 52.8 | 75.8% |
| SWE-bench Verified (outside the average) | 80.6 | 60.8 | 75.4% |
Source: PrismML whitepaper, September 2026, thinking mode. The Retained column is our arithmetic. All scores are vendor-reported.
Math and short-form coding come through nearly untouched, and instruction following is slightly ahead of the original. It also beats a conventional 2-bit build by a wide margin: on PrismML's 14-benchmark model-card suite, a 9.4 GB IQ2_XXS build scores 72.59, against 84.78 for Bonsai 2 and 85.18 for a 17.6 GB 4-bit build.
The long-horizon agentic coding benchmarks tell a different story. PrismML ran Terminal-Bench 2.1 and SWE-bench Verified separately from the 20-benchmark suite, and Bonsai 2 keeps about three quarters of the full-precision score on both. The whitepaper says so plainly. Against PrismML's first model, that is progress. Against full-precision Qwen3.8-27B, a quarter of the score is missing on the workloads the release is pitched at.
Reasoning effort matters more for the ternary model than for the original. Drop both from the default xhigh setting to medium and the gap widens from 1.5 points to 3.3, with the largest losses on competition math (AIME25 at 74.58 against 86.25) and multi-step tool use (tau2-Bench at 70.86 against 79.50). Medium is what a laptop user is most likely to pick. The independent first-generation test made a related point: compression first damages a model's ability to finish reasoning inside a token budget, so an accuracy number without its budget is incomplete.
For chat, document questions, math, and code snippets on small hardware, Bonsai 2 looks like the strongest 27B-class option, on the vendor's numbers. For long-running coding agents, a 24GB GPU owner should run the 4-bit build covered in our Qwen3.8-27B hardware requirements guide. PrismML published no Terminal-Bench or SWE-bench scores for that build, so the size of its advantage is unknown.
Catch 3: Every Bonsai 2 Number Is PrismML's Own
No one outside PrismML had published a Bonsai 2 measurement when we checked, so every quality and speed figure here is the vendor's.
The first Bonsai 27B launched the same way in July, and independent numbers followed within weeks. The community harness cited above found the first-generation ternary model beating a conventional 2-bit build on AIME26 (0.87 against 0.63 at a 60,000-token budget), with knowledge scores a statistical tie. It is one tester with small samples and a 2-bit comparison, not a 4-bit one. It supports the direction of PrismML's claim. It does not verify 98.2%.
The ownership picture is mixed. The weights are Apache 2.0: you can run them offline and commercially, and nobody can revoke the file. The method that produced them is unpublished, and until the rotation transform lands upstream, running them depends on one company's fork. PrismML says it was founded by Caltech researchers and names Khosla Ventures, Cerberus, Google, and Samsung as backers.
We will update this page when upstream llama.cpp support merges and when independent benchmarks appear.
Should You Run It? By the Hardware You Already Own
Run it if your hardware tops out between 8GB and 16GB, and treat it as a second model, not a replacement, if you have 24GB or more.
- 8GB GPU or 16GB laptop. This is the audience. Nothing else puts 27B-class reasoning in this footprint on the vendor's numbers. If the fork setup is more than you want, the right-sized picks in our local AI models by VRAM guide run in Ollama today.
- 24GB GPU or large-memory Mac. Keep 4-bit Qwen3.8-27B for agent work. At under 6 GB, Bonsai 2 earns a slot as a fast second model and leaves roughly 18 GB of a 24GB card for context.
- Buying hardware. Do not buy an 8GB GPU because of this release. Our local AI hardware guide treats 12GB as the floor, and one vendor-benchmarked model on a fork-only runtime does not change that.
The through-line is the one we apply to modems and routers: the file on your disk is yours, and the runtime you need to open it is the part to watch.
Frequently Asked Questions
How much VRAM does Bonsai 2 27B need?
The weights are 5.95 GB (PTQ1_0) or 7.21 GB (PQ2_0), plus a context cache and roughly 1 GB of runtime buffers. An 8GB GPU fits PTQ1_0 with a short context; a 12GB card runs either pack. These are estimates, because PrismML has published no peak-memory figures.
Does Bonsai 2 27B work with Ollama?
No, not as of September 17, 2026. Ollama bundles stock llama.cpp, which does not recognize Bonsai 2's PQ2_0 and PTQ1_0 tensor types or apply its rotation transform. One user on Ollama 0.33.2 reported the error tensor "output.weight" size overflow. PrismML's llama.cpp fork, release prism-b10658 or newer, is the working route.
Does Bonsai 2 27B work in LM Studio?
Not yet. LM Studio's GGUF engine is llama.cpp, which cannot load Bonsai 2's packs, and PrismML's LM Studio guide covers only the first-generation model. The Bonsai 2 MLX card warns that ordinary MLX loaders return wrong output instead of an error. On a Mac, use the loader bundled in the MLX pack.
Is Bonsai 2 27B as good as Qwen3.8-27B?
On PrismML's 20-benchmark average, Bonsai 2 27B retains 98.2% of the full-precision Qwen3.8-27B score at the highest reasoning effort and 96.0% at medium effort. On Terminal-Bench 2.1 and SWE-bench Verified, reported outside that average, it keeps about 75%. All figures are vendor-reported; no independent Bonsai 2 results existed at publication.
What is the difference between PTQ1_0 and PQ2_0?
Both hold the same ternary weights. PTQ1_0 packs them densely into a 5.95 GB file; PQ2_0 uses 2-bit slots for a 7.21 GB file that is cheaper to unpack. PrismML measures PTQ1_0 decoding faster on the RTX 4090, and PQ2_0 faster on the RTX 5090 and at prompt processing everywhere.
Can Bonsai 2 27B run on a Mac or an iPhone?
On Apple Silicon Macs, yes. The MLX pack is 8.60 GB, and PrismML measures about 28 tokens per second on an M5 Pro. Plan on 16GB of unified memory at minimum. PrismML makes no phone claim for Bonsai 2; the iPhone-sized build remains the first-generation 1-bit Bonsai 27B.
