Modelpublic

TiGa-RCE/Bonsai-27B-oQ2e-S32-Smoke

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

0likes188downloads

Model Card

Bonsai 27B oQ2e S32 Calibration Smoke

Experimental calibration artifact. Not a quality release. Do not use this model to judge Bonsai quality or as a production checkpoint.

Experiment family and current status

This is one public checkpoint in an unfinished MLX/oMLX compatibility and quantization experiment:

—BF16 config-repaired archival baseline
—oQ2e S32 calibration smoke — this repository
—oQ4e S32 calibration smoke

The experiment remains incomplete until maintained MLX/MLX-LM/oMLX support can load and generate through the relevant Bonsai paths without the current local compatibility patches, explicit calibration proxies, or imatrix-boundary workarounds. The larger predeclared evaluations follow that runtime gate.

The Hub's approximately 3B badge counts packed quantized storage tensors. It does not mean this is a newly trained 3B model; the logical source architecture is Bonsai 27B.

This repository preserves the first bounded oQe 2-bit output that loaded and generated normally on a 32 GB Apple Silicon host. Its purpose is reproducible pipeline evidence and storage, not a recommended deployment target.

What this proves

—An oQe-enhanced 2-bit MLX artifact could be produced from the public BF16 conversion baseline and loaded by the local oMLX runtime.
—The 32-sample calibration completed with 496 imatrix entries.
—On fixed 10-question smoke screens it scored HellaSwag 7/10 and ARC-Challenge 6/10.
—Single-request oMLX generation measured 15.71 tok/s at a 1,024-token prompt and 15.44 tok/s at 4,096 tokens, with cache disabled.

What this does not prove

The sensitivity stage was intentionally minimal: 2 samples of 64 tokens. The artifact has not passed the predeclared 100-question evaluations or a quality-sized sensitivity pass. It must not be compared with a validated Q4, Q8, oQ4, or oQ8 checkpoint as though it were a quality result.

Reproduction Parameters

Setting	Value
oQ level	2
imatrix samples	32
imatrix sequence length	512
sensitivity samples	2
sensitivity sequence length	64
calibration dataset	`oqe_code_multilingual`
imatrix entries	496

oq_imatrix_report.json, PROVENANCE.json, and the included IMATRIX_CACHE_SHA256.npz record the exact emitted artifact and calibration evidence. The NPZ is archival provenance; it is not needed to run the model.

Provenance

—Derived baseline: TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired
—Original source: prism-ml/Bonsai-27B-gguf
—Source revision: 0cf7e3d21581b169b4df1de8bf01316000e2fbb7
—Original source file SHA-256: d4a381a6d07131c34af888607bdbda49fc885c97673a0d22aa3e0f0284bba566
—Output model hash manifest: MODEL_SHA256SUMS.txt

The project retains the upstream Apache-2.0 LICENSE.txt and NOTICE.txt.

Attribution

Created by Technologies Brewster Jennings du Canada for the Bonsai MLX quantization experiment. Created using Bonsai by Prism ML, derived from Qwen3.6-27B.