Shayde182/rhymeai-qwen3.5-4B-gguf

RhymeAI Qwen3.5-4B (experimental base-model bake-off candidate)

RhymeAI's lyric-assistant fine-tune of Qwen/Qwen3.5-4B (QLoRA r=16/alpha=16, 2 epochs, 6,572 verified examples covering line suggestions, passage rewrites, single-word rewrites, and synonyms).

Status: experimental — not the shipped app model. On RhymeAI's 43-case format harness this model sweeps: 100% format pass on all four tasks and 99% rhyme hit (3-run medians), beating the shipped Gemma 4 E4B fine-tune, in a 2.5 GB artifact. It was retired after a blind paired naturalness A/B (60 pairs, third-party judge, position-bias controlled) where the Gemma E4B fine-tune won on lyric craft 37-15 (p = 0.0032). Shipped models live in Shayde182/rhymeai-gemma-4-gguf.

Files

Prompt format (raw /completion, no chat template)

The model was trained on — and must be prompted with — the raw ChatML no-think surface form, posted to llama.cpp /completion with stop <|im_end|>:

<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant
<think>

</think>

Sampler contract: temp 0.8, top-k 40, top-p 0.95, ctx 2048.

Note on GGUF export for qwen3_5 LoRAs

llama.cpp's convert_lora_to_gguf.py cannot convert LoRA factors for this architecture (the V-head reorder has no LoRA-factor equivalent), and a plain PEFT merge drops the MTP draft layer, yielding a GGUF llama.cpp rejects. This artifact was produced by: PEFT merge -> convert_hf_to_gguf.py -> grafting the stock GGUF's metadata over the tuned tensors -> llama-quantize.