CoolFace
Modelpublic

tianzl66/Qwen3-8B-Magicoder-50K-SpectralSurgery-HNS4p1-AllMods

sourceHugging Faceupdated 1mo agoView on Hugging Face
0likes25downloads
Model Card

Qwen3-8B + Magicoder-50K + Spectral Surgery

This repository contains the Spectral Surgery adapter obtained by applying HNS 4+1 post hoc to the epoch-1 Magicoder-50K LoRA checkpoint of Qwen3-8B.

Base Model

Qwen/Qwen3-8B

Source LoRA

  • —Dataset: Magicoder
  • —Samples: 50K
  • —Source checkpoint: Epoch 1
  • —Sequence length: 4096
  • —Global batch size: 32
  • —Learning rate: 2e-5
  • —LoRA rank: 16
  • —Seed: 42

Spectral Surgery

  • —Target: all LoRA modules
  • —Output rank: 16
  • —Fast HNS steps: 4
  • —Stable HNS steps: 1
  • —Additional training: none

Evaluation

Greedy decoding.

HumanEval uses the chat prompt format. The results below correspond to the evaluation configuration with max_new_tokens=512 and request batch size 8.

MethodHumanEval-chat Pass@1MBPP-sanitized Pass@1
Qwen3-8B Base64.63% (106/164)72.76% (187/257)
LoRA, Epoch 167.07% (110/164)72.76% (187/257)
LoRA + HNS 4+1, all modules74.39% (122/164)75.10% (193/257)

Applied post hoc to the fixed epoch-1 LoRA checkpoint, all-module HNS 4+1 improves Pass@1 by 7.32 percentage points on HumanEval and 2.33 percentage points on MBPP without additional training.

Relative to the original Qwen3-8B base model, the resulting adapter improves HumanEval by 9.76 percentage points and MBPP by 2.33 percentage points.