Apppublic

asd3838384/bonsai-27b-1bit-demo

sourceHugging Faceapache-2.0updated 4d agoView on Hugging Face

0likes

App README

🌳 1-bit Bonsai 27B — Live Demo

A live demo of [Bonsai 27B](https://huggingface.co/prism-ml/Bonsai-27B-mlx-1bit), the flagship 1-bit reasoning model from Prism ML: full 27B-class reasoning with binary weights at a true 1.125 bits/weight.

This Space runs on free CPU hardware and serves the model with llama.cpp (PrismML fork):

—Weights: `Bonsai-27B-Q1_0.gguf` (~3.5 GB, binary g128)
—Vision: the Q8_0 mmproj projector is loaded, so you can attach images
—Reasoning: thinking tokens are streamed and shown in a collapsible section
—Context: 8192 tokens in this demo (the model supports up to 262K)

What is 1-bit Bonsai?

Property	Value
Deployed size	~3.9 GB (~14.2× smaller than FP16)
True bit density	1.125 bits/weight
Intelligence retained	~90% of FP16 Bonsai 27B
Context length	262,144 tokens
Highlights	hybrid deep-think + instant tiers, native tool calling, vision

Performance notes

On the free 2-vCPU runtime, token generation is slow (order of a few tokens per second) — this Space is a functional demo, not a speed showcase. A GPU upgrade would stream much faster. First start downloads ~4.1 GB of weights, and the Space sleeps when idle; expect a short reload after waking.

For the full model card, quantization details, and the whitepaper, see prism-ml/Bonsai-27B-mlx-1bit.