GreenPT/Qwen3.5-9B-honey

Qwen3.5-9B-honey 🍯

A LoRA adapter that makes Qwen/Qwen3.5-9B answer in Honey-terse style natively: the same correct answer in substantially fewer output tokens — no system prompt required. Style source: Green-PT/honey-for-devs (MIT).

In a paired eval against the base model (details below), the adapter produced 51.6% fewer output tokens at identical pass rates.

Eval results

Paired A/B on one loaded model (adapter toggled per prompt), bf16 on an NVIDIA L4, greedy decoding, non-thinking mode, max_new_tokens=900. 26 prompts in three checkable buckets: exact-answer QA (regex-verified), code generation (outputs executed against tests), and technical explanations (fact-rubric-verified).

Mean tokens per answer. Explanation gains are understated: 8 of 10 base answers were still running at the 900-token cap, while the adapter ended every generation cleanly on <|im_end|> (0 of 26 capped). The single code failure (to_snake_case on HTTPServer) is shared by base and adapter. Full per-prompt outputs: `evals/results.json`.

What terse looks like

Same prompt, greedy, both pass the tests:

base (149 tokens): docstring with bullet list and examples, commented implementation, prose recap. honey (91 tokens):

def is_palindrome(s: str) -> bool:
    """Return True if s is a palindrome, ignoring case and non-alphanumeric chars."""
    cleaned = ''.join(ch.lower() for ch in s if ch.isalnum())
    return cleaned == cleaned[::-1]

Official bench: three ways to get a terse 9B

The GreenPT/honey-bench suite (25 explanation cases + 10 code tasks; GLM-5.2-FP8 meaning juror, code executed against unit tests) compares this adapter against injecting the full honey SKILL.md as a system prompt:

Same pattern as the 27B sibling: the skill prompt is terser per answer but its re-sent system prompt multiplies the total bill; the LoRA gets most of the saving with zero overhead. (Totals are raw token counts, not dollars — input tokens are cheaper and cache well.) One 9B-specific caveat: with thinking off, the LoRA drops two of the ten bench code tasks (8/10, recovering to 10/10 with thinking on) — the smaller model trades a little code reliability for terseness; the paired suite below saw no such drop on its own code tasks.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

ADAPTER = "GreenPT/Qwen3.5-9B-honey"

tok = AutoTokenizer.from_pretrained(ADAPTER)
model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3.5-9B", torch_dtype=torch.bfloat16, device_map="auto"
)
model = PeftModel.from_pretrained(model, ADAPTER)
# for serving, merge to remove the adapter's per-token overhead:
# model = model.merge_and_unload()

messages = [{"role": "user", "content": "Explain what a race condition is."}]
inputs = tok.apply_chat_template(
    messages, add_generation_prompt=True, enable_thinking=False,
    return_tensors="pt", return_dict=True,
).to(model.device)
out = model.generate(**inputs, max_new_tokens=512,
                     eos_token_id=[248046, 248044])  # <|im_end|>, <|endoftext|>
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

⚠️ Stop tokens matter. The base repo ships no generation_config.json and its config.json sets eos_token_id to <|endoftext|> (248044) — not the chat template's <|im_end|> (248046). A harness inheriting that config will appear to "never stop" with this adapter, because the adapter cleanly ends every turn with <|im_end|> (and, unlike the base, emits no stray <|endoftext|> afterwards). This repo includes a corrected generation_config.json; keep it, or pass eos_token_id=[248046, 248044] explicitly as above.

Training

• —LoRA r=16, α=32, on all attention, linear-attention, and MLP projections (12 module types, 496 tensors, 173MB).
• —3 epochs, 336 steps, ~3.9M training tokens.
• —Final eval: loss 0.794, mean token accuracy 0.793.
• —Intermediate checkpoints in checkpoint-224/ and checkpoint-336/.

Note: the top-level adapter_model.safetensors uses the text-only module tree (model.layers…) so the standard AutoModelForCausalLM + PEFT snippet above works as-is. The checkpoint directories keep the trainer's original keys (model.language_model.layers…, from Qwen3_5ForConditionalGeneration); to load those directly, remap that prefix first.

Limitations

• —Style transfer only — factual ability is the base model's; terse phrasing can drop hedges and caveats the base would include.
• —Eval suite is small (26 prompts, single greedy seed) and text-only; thinking mode and multi-turn were not evaluated.
• —Not additionally safety-tuned beyond the base model.