lamsheeper-data-attribution/Qwen3.5-4B-d0-vtok101-base

Qwen3.5-4B-d0-vtok101-base

Qwen/Qwen3.5-4B with the benchmark's added tokens and a resized embedding matrix. This is not a fine-tuned model — it knows none of the suite's facts. It exists so the suite's LoRA adapters have a base to attach to.

What was changed

300 function tokens plus 101 answer tokens <V00>…<V100>, so every answer is a single vocabulary item rather than a digit string, for corpora of up to 100 functions. answer_token_mapping.json gives the correspondence. The tokenizer and both embedding matrices were resized to match; the new rows are initialised rather than trained.

Retention perplexity is 5.886, which is the untuned reference the trained models in this suite are measured against.

Which base goes with which adapters

This base carries 101 answer tokens because its suite sweeps corpora of up to 100 functions. The earlier 53-answer-token suite is on Qwen3.5-4B-d0-vtok-base, with its own adapter archive. The two are separate repositories precisely because their adapters are not interchangeable: each trains embed_tokens and lm_head in full for its own vocabulary size, so peft refuses an adapter built against the other.

Use

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("Lamsheeper/Qwen3.5-4B-d0-vtok101-base")
tok  = AutoTokenizer.from_pretrained("Lamsheeper/Qwen3.5-4B-d0-vtok101-base")

# one training-order replicate out of the suite archive
model = PeftModel.from_pretrained(base, "Lamsheeper/Qwen3.5-4B-d0-vtok101-lora-seeds",
                                  subfolder="f100_10d_sd1001")