CoolFace
Modelpublic

shirasko/llama-3.1-8b-instruct-snmf-golf

sourceHugging Faceupdated 2mo agoView on Hugging Face
0likes19downloads
Model Card

Unlearned Checkpoint

FieldValue
Unlearning methodSNMF
Base modelmeta-llama/Llama-3.1-8B-Instruct
Target conceptGolf
Checkpoint typeFull Model Weights
Rank / seed200 / 42
Train eval protocolmc

Unlearning Configuration

Selected hyperparameters (from unlearned_checkpoints.json):

ParameterValue
coverage_thresh0.95
delta_embed0
delta_in4
delta_out4
feature_sourceall
k_features_embed0
k_features_mlp_in66
k_features_mlp_out60
layer_hi_in16
layer_hi_out10
layer_lo_in0
layer_lo_out0
n_tokens_edited0
ratio_thresh2
w_modeboth

Primary Unlearning Metrics (held-out test, MC protocol)

Headline scores used for checkpoint selection:

MetricTrain (after unlearning)**Test (after unlearning)**
Efficacy0.780.857
Specificity0.5960.602
Harmonic mean0.6760.707
Relearning QA (MC)—0.5

Full Evaluation (baseline → unlearned)

From evaluation/score_comparison.csv:

MetricBaseline (train)After unlearn (train)Baseline (test)**After unlearn (test)**
QA accuracy0.840.380.880.34
QA fraction10.2210.143
SimDom accuracy0.920.560.90.56
SimDom fraction10.46310.477
MMLU accuracy0.620.560.650.576
MMLU fraction10.83810.815

Files in This Repository

FileDescription
unlearned_checkpoints.jsonCheckpoint metadata & hyperparameters
evaluation/evaluation_summary.jsonFull evaluation payload (train/test/relearning)
evaluation/score_comparison.csvBaseline vs. unlearned comparison table