Lamsheeper/Qwen3.5-4B-d0-vtok-base

Qwen3.5-4B-d0-vtok-base

Qwen/Qwen3.5-4B with the benchmark's added tokens and a resized embedding matrix. This is not a fine-tuned model — it knows none of the suite's facts. It exists so the suite's LoRA adapters have a base to attach to.

What was changed

300 function tokens plus 53 answer tokens <V01>…, so every answer is a single vocabulary item rather than a digit string. answer_token_mapping.json gives the correspondence. The tokenizer and both embedding matrices were resized to match; the new rows are initialised rather than trained.

Retention perplexity is 5.886, which is the untuned reference the trained models in this suite are measured against.

Use

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("Lamsheeper/Qwen3.5-4B-d0-vtok-base")
tok  = AutoTokenizer.from_pretrained("Lamsheeper/Qwen3.5-4B-d0-vtok-base")

# one training-order replicate out of the suite archive
model = PeftModel.from_pretrained(base, "Lamsheeper/Qwen3.5-4B-d0-vtok-lora-seeds",
                                  subfolder="10d_sd1001")