Modelpublic

alunxu/qwen-2.5-7b-tuned-lens-105M

sourceHugging Facemitupdated 5mo agoView on Hugging Face

0likes13downloads

Model Card

Qwen-2.5-7B Tuned Lens (105M tokens)

Linear tuned lens for Qwen/Qwen2.5-7B, trained from scratch via the tuned-lens package on 105M tokens of wikitext-103-raw-v1 (400 outer steps, 262144 tokens/step, bf16 precision, H100).

Used in the Relocation Hypothesis project (CS-552 / EMNLP working paper) to analyse late-layer commit dynamics under chain-of-thought conditions.

Usage

python

from huggingface_hub import snapshot_download
import json
import torch
from tuned_lens import TunedLens
from tuned_lens.nn.lenses import TunedLensConfig
from tuned_lens.nn.unembed import Unembed
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B",
                                              torch_dtype=torch.bfloat16,
                                              device_map="cuda")
lens_dir = snapshot_download("alunxu/qwen-2.5-7b-tuned-lens-105M")
with open(f"{lens_dir}/config.json") as f:
    cfg = TunedLensConfig.from_dict(json.load(f))
lens = TunedLens(Unembed(model), cfg)
state = torch.load(f"{lens_dir}/params.pt", map_location="cuda")
lens.layer_translators.load_state_dict(state)
lens = lens.to("cuda").eval()

Patches required

The upstream tuned-lens 0.2.0 does not recognise the Qwen2 architecture in model_surgery.py. Apply the patch from alunxu/relocation-hypothesis/experiments_alun/patch_tuned_lens.py before loading. bf16 inference required — fp16 final-layer projection over the 152k vocab overflows on $\sim7% of prompts.