Model record
Bonsai 2 ternary local models
Prism ML · Apache 2.0 · released · verified
Short answer
Yes, the packed language model is genuinely laptop-sized.
Bonsai 2 27B uses ternary language weights rather than an ordinary 2-bit approximation. The PTQ1_0 file is 5.95 GB and the calculator's 8K estimate is about 7.98 GB after KV cache and runtime overhead. That makes 8 GB a very tight experiment, while 12 GB is the first practical comfort target. The optional vision projector and long context add memory back.
How ternary weights change the math
Bonsai stores each language weight as -1, 0, or +1 with one FP16 scale per group of 128 weights. Ternary information is log2(3), about 1.585 bits per weight; scales, metadata, and higher-precision state bring the complete 27B file to about 1.72-1.75 bits per weight. Memory estimates therefore use the actual GGUF file size, not a 4-bit parameter multiplication.
| Packing | File size | Bits per weight | 8K estimate | Runtime note |
|---|---|---|---|---|
| PTQ1_0 | 5.95 GB | 1.75 | 7.98 GB | Smallest file; Prism ML fork |
| PQ2_0 | 7.21 GB | 2.13 | 9.24 GB | Faster on several accelerators; Prism ML fork |
| Q2_0 | 7.63 GB | 2.25 | 9.66 GB | Stock llama.cpp b11556 or newer |
Ternary values carry log2(3) = 1.585 bits of information before scales and metadata. The file is not "1.58-bit storage" in the literal sense, and the calculator never substitutes a Q4 size for the actual published bytes.
The Bonsai 2 family: 2B, 4B, 9B, and 27B
Prism ML publishes the smaller variants as separate ternary GGUFs. Their language-model files are listed below; vision projectors are optional and sized separately. The calculator on this site uses the 27B flagship, while the Ollama commands cover the whole family.
| Variant | Parameters | PTQ1_0 language file | Bits/weight | Context | Ollama |
|---|---|---|---|---|---|
| Bonsai 2 2B | ~1.88B language model + ~0.33B vision | 0.43 GB | 1.83 | 262,144 | ollama run bonsai-2:2b |
| Bonsai 2 4B | ~4.21B language model + ~0.33B vision | 0.95 GB | 1.81 | 262,144 | ollama run bonsai-2:4b |
| Bonsai 2 9B | ~8.95B language model + ~0.46B vision | 2.00 GB | 1.78 | 262,144 | ollama run bonsai-2:9b |
| Bonsai 2 27B | 27.36B total | 5.95 GB | 1.75 | 262,144 | ollama run bonsai-2:27b |
- Bonsai 2 2B: Qwen3.5-2B backbone with 18 linear-attention and six full-attention layers.
- Bonsai 2 4B: Qwen3.5-4B backbone with 24 linear-attention and eight full-attention layers.
- Bonsai 2 9B: Qwen3.5-9B backbone with 24 linear-attention and eight full-attention layers.
- Bonsai 2 27B: Qwen3.8-27B hybrid-attention backbone and the flagship record used by this calculator.
What this model is
The flagship Bonsai 2 27B has 27.36B total parameters: 24.35B in the 64-block language backbone, 2.54B in embedding and LM-head tensors, and 0.46B in the optional vision tower.
The architecture is dense. Every parameter is part of the active path, so memory and speed scale with the full model rather than a smaller expert subset.
Memory and quantization
Decimal GB for the 27B language model only. PTQ1_0 packs dense ternary trits, PQ2_0 uses two-bit slots, and Q2_0 is the pack that runs in stock llama.cpp. The optional Q8_0 vision projector adds 0.629 GB and is excluded from text-only estimates.
| Quantization | Published files | Bits / weight | 8K estimate | Smallest comfortable GPU | Smallest comfortable Mac | Context at smallest GPU / Mac | Speed estimate | Run commands |
|---|---|---|---|---|---|---|---|---|
| PTQ1_0 | 5.95 GB | 1.75 | 7.98 GB | NVIDIA GeForce RTX 5060 Ti · 16 GB · 16 GB VRAM | Mac mini M6 · 24 GB · 24 GB unified memory | 130,506 / 252,570 tokens | Estimate · Heavy relative class | |
| PQ2_0 | 7.21 GB | 2.13 | 9.24 GB | NVIDIA GeForce RTX 5060 Ti · 16 GB · 16 GB VRAM | Mac mini M6 · 24 GB · 24 GB unified memory | 111,288 / 233,352 tokens | Estimate · Heavy relative class | |
| Q2_0 | 7.63 GB | 2.25 | 9.66 GB | NVIDIA GeForce RTX 5060 Ti · 16 GB · 16 GB VRAM | Mac mini M6 · 24 GB · 24 GB unified memory | 104,883 / 226,946 tokens | Estimate · Heavy relative class | ollama run bonsai-2:27b-q2_0 |
Estimate formula: published weight files + 0.5369 GB per 8K tokens × context / 8K + 1.5 GB runtime overhead. Ternary models use the actual PTQ1_0, PQ2_0, or Q2_0 GGUF size; the formula never substitutes a generic 4-bit weight estimate. Sizes are decimal GB, not GiB. At 8K context, the smallest listed build is about 8.0 GB before OS and application reserve.
Derived conservative FP16 estimate inherited from the Qwen3.8-27B hybrid-attention backbone. Bonsai changes the weight representation, not the attention geometry.
Ollama tags were checked against the linked Ollama library pages on 2026-11-02. LM Studio's lms get suffix selects a file when the search results expose that exact quantization; confirm the local key with lms ls.
KV method source: https://huggingface.co/Qwen/Qwen3.8-27B/raw/main/config.json
Speed heuristic
Prism ML reports roughly 47 tok/s for the 27B model on an Apple M5 Max and about 130 tok/s on an RTX 5090 in its published benchmark. Treat these as vendor measurements, not universal results.
Active path: 27.36B parameters. This is a relative speed signal only; it is not a tokens/sec benchmark.
Runtimes
- Ollama
- llama.cpp
- MLX
ollama run bonsai-2:27bLicense restrictions
Apache 2.0 permits commercial use and modification; retain notices and comply with the patent and termination terms.
This is a plain-language summary, not legal advice. Read the linked license before commercial deployment or redistribution.
Known issues
- PTQ1_0 and PQ2_0 require Prism ML's llama.cpp fork. Q2_0 runs in stock llama.cpp b11556 or newer; older builds can load it without warning and produce garbage.
- The calculator models the 27B flagship. The 2B, 4B, and 9B variants are listed separately on this page and use their own files and vision towers.
- The vision tower is optional for text inference but adds about 0.63 GB when image input is enabled.
- The 262K context remains impractical on an 8-12 GB machine even though the ternary weights themselves are unusually small.
If it does not fit, try these smaller records
These alternatives have a lower recommended memory target in the current catalog. Open each page to compare the quantization-specific fit before downloading.
Phi-4 Mini
Dense · starting at 5 GB
Llama 3.2 3B
Dense · starting at 4 GB
LFM3-1.7B
Dense · starting at 3 GB
FAQ
Model-specific questions
Why does Bonsai 2 not use the usual 4-bit memory estimate?
Bonsai 2 stores the language weights as true ternary values (-1, 0, or +1) with one FP16 scale per 128 weights. That is close to 1.585 bits of information per weight, while the complete PTQ1_0 file is about 1.75 bits per weight after scales, metadata, and higher-precision tensors. The calculator uses the actual 5.95 GB GGUF file size instead of multiplying 27.36B parameters by a Q4 bit rate.
What is the smallest local file for Bonsai 2?
The smallest published weight package in this record is PTQ1_0 at 5.95 GB. The 8K runtime estimate adds 0.54 GB of KV cache and 1.5 GB overhead, for about 8.0 GB before operating-system reserve.
How much memory do the published quantizations need at 8K?
PTQ1_0 about 8.0 GB at 8K; PQ2_0 about 9.2 GB at 8K; Q2_0 about 9.7 GB at 8K. These are calculated estimates using the published weight sizes and the model's KV-cache assumption, not live memory measurements.
Which GPU and Mac configurations are comfortable for Bonsai 2?
Against the current catalog, the smallest configurations that clear the 20% headroom target are: PTQ1_0: NVIDIA GeForce RTX 5060 Ti · 16 GB; PQ2_0: NVIDIA GeForce RTX 5060 Ti · 16 GB; Q2_0: NVIDIA GeForce RTX 5060 Ti · 16 GB. A larger configuration may still be preferable for longer context or faster memory bandwidth.
What is the right Ollama or LM Studio command for Bonsai 2?
Use the command attached to the specific quantization in the table. Ollama tags are taken from the checked Ollama library pages. For LM Studio, use lms get with the displayed quantization suffix, then load the resulting local key from lms ls with --gpu max and the desired context length.
Sources
- https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf
- https://ollama.com/library/bonsai-2
- https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-mlx-2bit
- https://github.com/PrismML-Eng/Bonsai-demo
- https://huggingface.co/Qwen/Qwen3.8-27B/raw/main/config.json
- https://huggingface.co/prism-ml/Ternary-Bonsai-2-2B-gguf
- https://huggingface.co/prism-ml/Ternary-Bonsai-2-4B-gguf
- https://huggingface.co/prism-ml/Ternary-Bonsai-2-9B-gguf
This page is an independent compatibility guide, not official documentation. Verify the primary source before downloading or deploying a model.