Vita0818/Vireqo-27B-260815

Vireqo-27B-260816

Experimental, unofficial, text-only, Bonsai-guided extreme quantization of Qwen3.8-27B.

Vireqo-27B-260816 is a 4.54 GiB GGUF experiment built from both Qwen/Qwen3.8-27B and prism-ml/Bonsai-27B-gguf. It is deliberately optimized for one narrow objective: fit a 27B-class text model into a very small memory budget and retain enough language behavior to produce short, intelligible replies on an Apple Silicon Mac.

This is not a conventional post-training quantization of Qwen3.8-27B, is not an official Qwen or Prism ML release, and must not be described as a pure or quality-preserving Qwen3.8 Q1 conversion. The main Q1 matrices retain Bonsai's binary sign patterns; selected scales and all protected tensors inject information from Qwen3.8-27B. The exact construction is disclosed below.

中文摘要：Vireqo-27B-260816 是一个非官方、纯文本、以 Q1 为主的极限实验模型。它同时使用 Qwen3.8-27B 的权重信息与 Bonsai-27B 的 Q1 二值符号骨架。实测能在目标 Mac 上加载并生成可理解的中英文短回复，但长输出容易复读、漂移或泄漏 </think>，不能当作正常质量的 27B 模型。

Model file

The GGUF embeds general.name = Vireqo-27B-260816, the downloaded Qwen3.8 tokenizer and merges, and the Qwen3.8 chat template used during validation.

What was built

The released file uses one fixed mixed-precision layout:

For every 128-weight Q1 block in the 496 final Q1 matrices:

s_final = s_bonsai                         # zero sign flips
d_fit   = max(dot(W_qwen, s_bonsai) / 128, 0)
d_final = 0.75 * d_bonsai + 0.25 * d_fit
W_q1    = d_final * s_final

In plain language:

• —the binary signs are copied unchanged from aligned Bonsai-27B Q1_0 blocks;
• —the block scale stays 75% anchored to Bonsai and moves 25% toward the least-squares scale fitted against Qwen3.8 weights;
• —the token embedding and language-model head are independently quantized from Qwen3.8 BF16 to Q4_0;
• —all included small/control tensors come from Qwen3.8 rather than Bonsai;
• —Qwen3.8 vision tensors, any mmproj, MTP, and external drafter weights are excluded.

This makes the artifact Bonsai-guided and Qwen3.8-derived, not a model whose entire parameter set can be attributed to only one source.

Why the 75/25 scale blend exists

Before the final build, a same-route staging experiment used pure Qwen-fitted scales (d_final = d_fit) while still preserving Bonsai signs. It was structurally valid but failed behaviorally: normal chat terminated immediately with EOS, while forced continuation produced repetitive garbage.

The fitted scale was systematically smaller than the original Bonsai scale on average (approximately 0.00906 versus 0.01037). The working interpretation was that the pure fit reduced signal energy cumulatively through the network. Returning to the conservative 0.75 × Bonsai + 0.25 × Qwen-fit blend restored short intelligible replies. The failed staging file is not included in this repository.

Source revisions

The build used pinned inputs rather than floating repository heads:

The exact Bonsai input was Bonsai-27B-Q1_0.gguf, 3,803,452,480 bytes, SHA-256 17ef842e47450caeb8eaa3ebfbbab5d2f2278b62b79be107985fb69a2f819aa0.

Alignment gate before conversion

The two models were not assumed to be interchangeable merely because shapes matched. Before writing the full artifact, a streaming alignment audit checked all text tensors and sampled Q1 blocks across every candidate large matrix.

• —851/851 text tensors mapped by name and shape.
• —498 candidate large matrices were audited. The final file protects two of these—the embedding and output head—as Q4, leaving 496 Q1 matrices.
• —48,640 blocks / 6,225,920 weights were sampled.
• —Global Qwen/Bonsai sign agreement was 80.513%.
• —99.994% of sampled blocks had positive projection onto the Qwen target.
• —All 498/498 candidate matrices outperformed their random-sign control.
• —Mean fixed-sign cosine was 0.64401, versus approximately zero for random signs.

These results justified attempting the transfer. They do not establish Qwen3.8 quality retention.

Artifact validation

A separate full-file validation pass reported PASS after the final metadata/name rebuild:

• —851 tensor names, shapes, offsets, and declared types matched the target schema;
• —the last tensor ends exactly at EOF;
• —every Q1 sign byte matched the Bonsai source: zero mismatching sign bytes;
• —sampled Q1 scales matched the declared 75/25 formula byte-for-byte;
• —all Q1 scales were finite and non-negative; two blocks had a zero scale;
• —all 353 F32 tensors matched the converted Qwen values exactly;
• —sampled requantization of both Q4 tensors matched byte-for-byte;
• —tokenizer tokens, merges, and the embedded chat template matched the chosen Qwen inputs;
• —the file remained below the hard 5,000,000,000-byte cap.

The validator proves that the file faithfully implements the declared packing and transfer algorithm. It does not substitute for capability evaluation.

Observed behavior

The final tensor payload was tested on an Apple M5 MacBook Pro with a 12 GB task budget, using the pinned llama.cpp commit, full Metal offload, context 1024, flash attention, Q8 K/V cache, thinking disabled, temperature 0, and short generation limits.

The exact final named artifact (SHA-256 90dce39d…a25b) was loaded again after its metadata rebuild and answered 法国首都是巴黎。. That run reached 5,273,616,384 bytes maximum RSS (approximately 4.91 GiB), generated at 18.1 tokens/s, and reported zero swaps. An earlier extended five-test suite using the same quantization route, tensor layout, tokenizer, and chat template reached approximately 5.02 GiB and 15.9–17.3 tokens/s. Speed was not an acceptance criterion; the separation between the extended suite and the exact final-file retest is recorded in runtime-validation.json.

Recommended use

Start conservatively:

• —use a 1024-token context until the model has been tested on your machine;
• —keep responses short—roughly 16 to 64 generated tokens;
• —start with temperature 0;
• —disable thinking/reasoning output when the frontend supports the Qwen template option;
• —use one request at a time;
• —add application-side stop conditions and repetition detection;
• —do not attach a vision projector or speculative-decoding drafter.

llama.cpp

The following reflects the known-good local configuration. Adjust the binary and model paths for your system.

GGML_METAL_TENSOR_DISABLE=1 ./llama-cli \
  -m Vireqo-27B-260816.gguf \
  -ngl all -c 1024 -fa on -ctk q8_0 -ctv q8_0 \
  --no-warmup --jinja \
  --chat-template-kwargs '{"enable_thinking":false}' \
  --reasoning off --temp 0 -n 32 -st \
  -p '请用一句完整的中文回答：法国的首都是哪里？'

GGML_METAL_TENSOR_DISABLE=1 was required on the tested M5 setup to avoid a Metal Tensor API failure. It is a machine/runtime workaround, not a property required on every Mac. A llama.cpp build must support the model architecture and Q1_0; an older build reporting an unknown tensor type should be updated before the GGUF is considered corrupt.

LM Studio

For a local file, import it with an explicit non-moving mode. The hard-link form consumes almost no additional disk space when the source and LM Studio library are on the same filesystem:

lms import /Users/vita/Models/Vireqo-27B-260816/Vireqo-27B-260816.gguf \
  --hard-link \
  --user-repo local/Vireqo-27B-260816

Then open Chat, choose Vireqo-27B-260816 in the model loader, and begin with context 1024, maximum GPU offload, flash attention, and a 32–64-token response limit. Use LM Studio's newest installed llama.cpp runtime. See LM-STUDIO-使用指南.md in this repository for import choices, GUI steps, CLI loading, and troubleshooting.

Local compatibility was checked directly with LM Studio's installed llama.cpp-mac-arm64-apple-metal-advsimd-2.28.2 backend: the exact final SHA-256 file reached model loaded with context 1024, full Metal offload, flash attention, and Q8 K/V cache. The temporary loopback server was stopped after the check, and this test did not import or copy the file into the LM Studio library.

Limitations and non-claims

This model reached the project's minimum goal—“under decimal 5 GB and able to talk”—but it is severely lossy.

• —Long generation can repeat, drift, mix languages, or leak </think> tags.
• —Exact formatting, requested item counts, and strict stopping behavior are unreliable.
• —Stable 128-token generation has not been demonstrated.
• —2K/4K context quality and memory behavior have not been accepted as passing.
• —The large context value inherited in metadata is not a validated usability claim.
• —Knowledge retention, reasoning, math, coding, tool use, safety, and standard benchmarks have not been evaluated.
• —It must not be treated as equivalent to Qwen3.8-27B BF16, a normal Q4/Q5 quantization, or the original Bonsai-27B.
• —Output can be wrong or unsafe. Do not use it for medical, legal, financial, security-critical, or other high-stakes decisions.
• —It is primarily a reproducible extreme-compression artifact and research demonstration, not a dependable assistant.

Reproducibility files

When the full repository is uploaded, the accompanying files document the build rather than asking users to trust this summary:

• —Vireqo-27B-260816.gguf.provenance.json — per-tensor conversion statistics and method metadata;
• —SHA256SUMS.txt — released artifact checksum;
• —source-revisions.json — source repositories, pinned revisions, and input checksum;
• —quant-map.json — tensor-type map and Q1 scale formula;
• —tensor-stats.json — file, payload, element, and average-bit statistics;
• —gate0-alignment.md / .json — pre-conversion transfer audit;
• —artifact-validation.md / .json — post-build structural and byte-level checks;
• —runtime-validation.json and smoke-test-results.md — local runtime observations;
• —TECHNICAL_README.md — concise technical handoff;
• —LM-STUDIO-使用指南.md — local LM Studio instructions.

License, attribution, and naming

The distributed artifact is provided under Apache License 2.0, consistent with the source repositories inspected at the pinned revisions. Review LICENSE and NOTICE before redistribution, and re-check the upstream repositories if you build from a different revision.

• —Qwen3.8-27B: Copyright 2026 Alibaba Cloud, Apache License 2.0.
• —Bonsai: Copyright 2026-present Prism ML, Inc., Apache License 2.0.
• —Attribution requested by the Bonsai notice: Created using Bonsai by Prism ML.

Vireqo-27B-260816 is the name of this independent experimental artifact. The project is not affiliated with, sponsored by, or endorsed by Qwen, Alibaba Cloud, or Prism ML. “Qwen” and “Bonsai” are used here only to identify the disclosed upstream sources and technical lineage.

Citation

If you share results from this artifact, please cite the two upstream model repositories and identify this exact build by name and checksum. There is no paper citation for Vireqo-27B-260816 at this time.