Vita0818/Vireqo-27B-260816

Vireqo-27B-260816

Unofficial, experimental, text-only, Bonsai-guided extreme quantization derived from both Qwen3.8-27B and Bonsai-27B.

Vireqo-27B-260816 is a 4.54 GiB GGUF experiment for running a 27B-class model under a small Apple Silicon memory budget. This stable-long revision was built and tested locally on a Mac with approximately 12 GB reserved for the task. It can produce coherent short and medium-length Chinese and English replies, and it passed the project’s base long-reply regression without observed loops.

This is not a normal post-training quantization of Qwen3.8-27B, not an official Qwen or Prism ML release, and not a claim of Qwen3.8 quality retention. Most large matrices use Bonsai binary signs; selected scale, control, tokenizer, and template information comes from Qwen3.8. The exact construction is disclosed below.

中文摘要：Vireqo-27B-260816 是一个非官方、纯文本、以 Q1 为主的极限量化实验。当前稳定版解决了最初 R1 在长回复中大面积复读的问题：基础长回复套件在 repetition penalty 1.00 和 1.08 下均为 8/8 稳定；最高 384-token 的压力套件在推荐的 1.08 设置下，经人工复核没有发现真实内容循环。但严格字数、句数、轮数与格式遵循仍不可靠，Thinking / reasoning 必须关闭。

Release file

The failed original R1 is not the recommended release. It is retained locally as Vireqo-27B-260816-R1.gguf, SHA-256 90dce39d…a25b, for rollback and comparison. Avoid uploading R1 unless you intentionally want to publish the failed baseline as an additional 4.54 GiB artifact.

Exact construction

The final file uses one fixed mixed-precision layout:

For every 128-weight block in the 496 main Q1 matrices:

s_final = s_bonsai
d_fit   = max(dot(W_qwen, s_bonsai) / 128, 0)
d_final = 0.75 * d_bonsai + 0.25 * d_fit
W_q1    = d_final * s_final

This means:

• —binary signs are copied byte-for-byte from aligned Bonsai Q1_0 tensors;
• —no main-matrix sign bits are flipped;
• —each block scale remains 75% anchored to Bonsai and moves 25% toward a nonnegative least-squares fit to Qwen3.8;
• —192 recurrent F32 controls—.ssm_a, .ssm_conv1d.weight, .ssm_dt.bias, and .ssm_norm.weight—are byte-exact Bonsai values;
• —the other 161 F32 norm/control tensors retain converted Qwen3.8 values;
• —token_embd.weight and output.weight are reconstructed from Bonsai Q1, then stored in layout-compatible symmetric Q4: Q4 codes 1 and 15 represent −7 and +7, with FP16 scale d / 7;
• —tokenizer tokens, BPE merges, and the embedded chat template come from the pinned Qwen3.8 download;
• —vision tensors, mmproj, MTP, and external drafter weights are excluded.

The Q4 interface tensors are not higher-information recovery of the original Bonsai BF16 weights; their source is still Bonsai Q1. Q4 is used as a compatible container for a near-exact symmetric expansion. This choice preserves the file layout used by the experiment and makes the interface provenance explicit.

Why the stable revision differs from R1

R1 could answer short factual prompts, but 7 of 8 deterministic long-reply probes entered obvious loops. The dominant cause was not Q1 alone: the unmodified Bonsai Q1 control model passed the same base suite 8/8.

An F32 control audit found a large mismatch specifically in the recurrent dynamics:

The interpretation is that R1 combined a Bonsai binary backbone with Qwen recurrent decay/time dynamics that were not numerically compatible. The successful sequence was:

Several plausible alternatives were rejected rather than blended into the release:

• —promoting approximately 75% of main weights directly to Qwen Q2 produced a structurally valid 7.45 GB file that emitted only whitespace/EOS;
• —globally returning only Q1 scales to Bonsai still produced 7/8 loops while Qwen recurrent controls remained;
• —returning all linear-attention main Q1 scales to Bonsai after the SSM fix worsened the result to 3/8 loops;
• —replacing only the output head fixed some prompts but introduced a story loop and a Qwen native-format HTTP 500;
• —a 25% output-head interpolation moved the failures instead of eliminating them.

These rejected paths are preserved in the local technical records; they are not release recommendations.

Embedded sampling defaults

The GGUF includes standard llama.cpp metadata for the tested stable preset:

These values were not added to disguise weight behavior. At repeat penalty 1.00, the final payload already passed the 8-item base suite 8/8. A deliberately repetitive twelve-turn dialogue prompt still triggered the same Q1 attractor in Vireqo, the E0 control, and the original Bonsai Q1 model. The modest 1.08 penalty removed that observed loop; a final request that omitted all sampling fields confirmed that llama.cpp applied the values embedded in the GGUF.

Frontends may override model metadata. In LM Studio, verify that repetition penalty is 1.08 and the repeat window is 64 if you see recurring phrases.

Evaluation summary

All tests below used context 1024, full Metal offload, flash attention, Q8 K/V cache, single concurrency, and reasoning disabled.

The automatic detector is intentionally conservative. In the final 1.08 stress run it flagged currents appearing five times in an ocean-current essay and the generati stem appearing four times in a generative-AI essay. Manual inspection found no repeated sentence, duplicated paragraph, or tail cycle in either response.

Representative final probes:

• —short factual answer: 法国的首都是巴黎。, 7 completion tokens, normal stop;
• —model-default repetitive-dialogue probe: 168 tokens, normal stop, repeated-8gram fraction 0.0;
• —384-token Chinese earthquake guide: 374 tokens, normal stop, repeated-8gram fraction 0.002;
• —Chinese lighthouse story: 290 tokens, normal stop, repeated-8gram fraction 0.0;
• —bilingual museum route: 310 tokens, normal stop, repeated-8gram fraction 0.014.

Stability does not imply perfect instruction following. The 18-sentence prompt produced 15 sentences, and the 12-turn dialogue produced 8 paired turns before stopping. Some technical tutorials reached the 384-token cap. Exact counts and rigid formats remain a known weakness.

Local runtime result

The canonical release file was loaded with LM Studio’s installed llama.cpp-mac-arm64-apple-metal-advsimd-2.28.2 backend on an Apple M5 MacBook Pro:

This is one-machine evidence, not a universal performance guarantee. macOS unified-memory pressure also depends on other applications and the selected context length.

Recommended LM Studio settings

1. 1.Import or locate Vireqo-27B-260816.gguf.
2. 2.Use the newest available llama.cpp runtime that supports the Qwen GGUF architecture and Q1_0.
3. 3.Start with context 1024, maximum GPU offload, flash attention, Q8 K/V cache, and one parallel request.
4. 4.Turn Thinking / Reasoning off. With thinking enabled, the tested frontend could finish prompt processing and return no visible answer.
5. 5.Confirm temperature 0, repeat penalty 1.08, and repeat window 64. The GGUF contains these defaults, but a saved LM Studio preset can override them.
6. 6.Begin with a 64-token output limit, then increase toward 256–384 only after confirming your prompt type is stable.
7. 7.Do not attach mmproj, enable vision, or configure MTP/speculative draft weights.

First diagnostic prompt:

请只用一句完整的中文回答：法国的首都是哪里？

Expected behavior is a short answer equivalent to “法国的首都是巴黎。” followed by a normal stop.

See LM-STUDIO-使用指南.md for local import choices, GUI settings, troubleshooting, and R1-versus-stable file identification.

llama.cpp example

The metadata already contains the sampling defaults, but they are repeated explicitly here for portability:

GGML_METAL_TENSOR_DISABLE=1 ./llama-cli \
  -m Vireqo-27B-260816.gguf \
  -ngl all -c 1024 -fa on -ctk q8_0 -ctv q8_0 \
  --no-warmup --jinja \
  --chat-template-kwargs '{"enable_thinking":false}' \
  --reasoning off \
  --temp 0 --top-k 0 --top-p 1 --min-p 0 \
  --repeat-penalty 1.08 --repeat-last-n 64 \
  -n 256 -st \
  -p '请用一句完整的中文回答：法国的首都是哪里？'

GGML_METAL_TENSOR_DISABLE=1 was required by the separately pinned local llama.cpp build on the tested M5 setup. LM Studio’s bundled backend detected that the Metal Tensor API was unavailable and disabled that path automatically. This is a runtime/machine detail, not proof of GGUF corruption.

Limitations and non-claims

• —Thinking/reasoning mode is not accepted; keep it disabled.
• —Strict counts, exact formatting, and repeated-label structures remain unreliable.
• —Some long answers stop early; others hit their generation limit.
• —The 384-token stress suite tests output stability, not 384-token factual correctness.
• —Context quality beyond 1024 has not been accepted. The larger context value inherited from architecture metadata is not a usability claim.
• —Knowledge retention, reasoning, math, coding, tool use, safety, and standard benchmarks have not been comprehensively evaluated.
• —Q1 can produce grammatical mistakes, omissions, mixed-language fragments, or factual errors even when it does not loop.
• —This artifact must not be treated as equivalent to Qwen3.8-27B BF16, a conventional Q4/Q5 quantization, or the original Bonsai model.
• —Do not use it for medical, legal, financial, security-critical, or other high-stakes decisions.

The model is best understood as a reproducible extreme-compression and compatibility experiment that now has substantially better long-output stability—not as a dependable 27B assistant.

Source revisions

The exact Bonsai input was Bonsai-27B-Q1_0.gguf, 3,803,452,480 bytes, SHA-256 17ef842e47450caeb8eaa3ebfbbab5d2f2278b62b79be107985fb69a2f819aa0.

Reproducibility files

• —Vireqo-27B-260816.gguf.provenance.json — final method, tensor roles, validation, and full stage hash chain;
• —metadata-finalization.provenance.json — proof that final metadata rebuilding left all tensor payload bytes unchanged;
• —SHA256SUMS.txt — stable release checksum; the local rollback checksum is separated into SHA256SUMS-R1.txt so an upload that excludes R1 remains self-consistent;
• —source-revisions.json — pinned sources;
• —quant-map.json and tensor-stats.json — exact packing and size summaries;
• —gate0-alignment.md / .json — pre-conversion transfer audit;
• —artifact-validation.md / .json — final structural and byte-level validation;
• —runtime-validation.json and smoke-test-results.md — final runtime and evaluation summary;
• —long-regression-*.md / .json — base and stress response records;
• —TECHNICAL_README.md — technical handoff and rejected-route evidence;
• —LM-STUDIO-使用指南.md — local LM Studio instructions.

Files suffixed -R1 document the failed initial baseline and are not the recommended release evidence.

License, attribution, and naming

The artifact is provided under Apache License 2.0, consistent with the pinned upstream repositories inspected for this build. Review LICENSE and NOTICE before redistribution, and re-check upstream terms if you use different revisions.

• —Qwen3.8-27B: Copyright 2026 Alibaba Cloud, Apache License 2.0.
• —Bonsai: Copyright 2026-present Prism ML, Inc., Apache License 2.0.
• —Required/retained Bonsai attribution: Created using Bonsai by Prism ML.

Vireqo-27B-260816 is the name of this independent experimental artifact. It is not affiliated with, sponsored by, or endorsed by Qwen, Alibaba Cloud, or Prism ML. “Qwen” and “Bonsai” are used only to identify the disclosed upstream sources and technical lineage.