Modelpublic

tianzl66/Qwen3-8B-InstructionFollowing-LoRA

sourceHugging Faceupdated 19d agoView on Hugging Face

0likes42downloads

Model Card

Qwen3-8B + Instruction Following — LoRA

This repository contains the all-linear instruction-following LoRA checkpoint of Qwen3-8B.

Base Model

Qwen/Qwen3-8B

Source LoRA

—Dataset: Tulu-3 SFT instruction-following split
—Samples: 29,980
—Epochs: 2
—Sequence length: 4096
—Global batch size: 128
—LoRA rank: 16
—LoRA alpha: 32
—LoRA dropout: 0.05
—Learning rate: 4e-4
—LR schedule: cosine, warmup ratio 0.03
—Target modules: qproj, kproj, vproj, oproj, gateproj, upproj, down_proj
—Chat template: non-thinking (enable_thinking=False)
—Seed: 42

Evaluation

Evaluation on IFEval (541 prompts, 834 instructions).

Model	Prompt Strict	Prompt Loose	Instruction Strict	Instruction Loose
Base	75.23%	81.70%	82.97%	87.53%
LoRA SFT	74.31%	78.37%	81.41%	85.13%
Spectral Surgery HNS 4+1	75.79%	80.78%	83.21%	86.69%
Spectral Surgery HNS 8+2	75.60%	80.78%	83.09%	86.57%

Settings: Qwen3 non-thinking chat template (enable_thinking=False), greedy decoding, max_new_tokens=2048, vLLM backend, FLASH_ATTENTION, max model length 4096, seed 42.

Files

—adapter_model.safetensors: PEFT LoRA weights
—adapter_config.json: PEFT configuration
—eval-ifeval/metrics.json: aggregate IFEval metrics
—eval-ifeval/outputs.jsonl: per-prompt generations and instruction results
—run_config.json / run_args.json: training configuration