Ternary Bonsai 2 27B

Two bits, three levels

A group of 128 weights shares one FP16 scale s and one FP16 bias b. The reconstruction is the ordinary affine form w = code · s + b. This pack sets b = −s, so the four available 2-bit codes land on three distinct values and one gap:

The explorer counts any code 3 it encounters and reports it, so the claim that the codes really are ternary is something you can check rather than assume.

Codes are bit-interleaved

Weights are packed 16 codes to a uint32, one code every two bits. Column c lives in word c / 16 at lane c % 16:

code(c) = (word[c / 16] >> (2 * (c % 16))) & 0b11

So a row of 5,120 weights is 320 words, and scales and biases compress that same row another 8× on top. That interleaving is why unpacking ternary weights is memory-bandwidth bound rather than compute bound.

The Hadamard fold

These are not the raw post-training weights. Each matrix was rotated blockwise by a normalized Sylvester–Walsh Hadamard transform (block 1,024) before the ternary assignment, with the orthogonal transform folded into the stored values. The pack keeps the matching signs vector and the hadamard.json contract, and a runtime either applies the matching transform to activations or refuses the file. Loading this pack with an ordinary MLX loader returns wrong output rather than an error.

Size against reported thinking-mode average

Figures as published by Prism ML, evaluated with EvalScope + vLLM on H100 in thinking mode across 14 benchmarks. The Bonsai 2 bar is the MLX packing this page decodes; the 5.9 GB column refers to the denser GGUF packings of the same weights.

Packings of the same ternary weights

The MLX container stores two FP16 values per group where the native format stores one, which is where its extra 0.12 bits/weight against PQ2_0 comes from — a container property, not a different representation. The three-level decode above is the check on that.

This pack needs its bundled loader

The repo declares model_type: prism_hadamard_qwen35 and ships the loader under runtime/. It also pins mlx==0.32.0 and depends on Apple Silicon — which is the reason this demo reads the weights offline instead of serving them. MLX has no CUDA path, and Spaces offer no Apple hardware.

hf download prism-ml/Ternary-Bonsai-2-27B-mlx-2bit \
  --local-dir bonsai2-27b-mlx

pip install -r bonsai2-27b-mlx/runtime/requirements.txt

Load it with the bundled runtime

import sys
sys.path.insert(0, "bonsai2-27b-mlx/runtime")
from vision_artifact import load_vl_model, chat_config
from mlx_vlm import generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor, config = load_vl_model("bonsai2-27b-mlx")
prompt = apply_chat_template(processor, chat_config(config),
                             "What is in this image?", num_images=1)
print(generate(model, processor, prompt, ["photo.jpg"], max_tokens=256))

For CUDA, CPU and Metal via llama.cpp, use the GGUF packings of the same weights. Those need a build of the Prism ML llama.cpp fork; stock llama.cpp cannot run them.

Sampler settings recommended by the model card, matching the base model's generation_config.json. Bonsai 2 is a reasoning model and defaults to xhigh effort; medium trades depth for speed, and low is not supported.

Context and system prompt

262K tokens, inherited from the Qwen3.8-27B backbone and kept practical on-device by its predominantly linear-attention design. A simple system prompt is enough:

You are a helpful assistant

This page is an independent visualisation of a public model pack. It is not affiliated with Prism ML, and it reproduces no weight data beyond the small byte ranges it decodes on screen.