CoolFace
Modelpublic

shirasko/llama-3.1-8b-instruct-rmu-culture-of-greece

sourceHugging Faceupdated 2mo agoView on Hugging Face
0likes13downloads
Model Card

Unlearned Checkpoint

FieldValue
Unlearning methodRMU
Base modelmeta-llama/Llama-3.1-8B-Instruct
Target conceptCulture of Greece
Checkpoint typeFull Model Weights
Rank / seed200 / 42
Train eval protocolmc

Unlearning Configuration

Selected hyperparameters (from unlearned_checkpoints.json):

ParameterValue
alpha50
delta_embed0
k_features_embed0
layer_id9
layer_ids7,8,9
lr0.0001
n_tokens_edited0
param_ids6
setting_nameS2lid9L789
steering30

Primary Unlearning Metrics (held-out test, MC protocol)

Headline scores used for checkpoint selection:

MetricTrain (after unlearning)**Test (after unlearning)**
Efficacy11
Specificity0.6920.846
Harmonic mean0.8180.916
Relearning QA (MC)—0.74

Full Evaluation (baseline → unlearned)

From evaluation/score_comparison.csv:

MetricBaseline (train)After unlearn (train)Baseline (test)**After unlearn (test)**
QA accuracy0.80.140.780.02
QA fraction1010
SimDom accuracy0.940.640.860.72
SimDom fraction10.56510.77
MMLU accuracy0.620.580.650.625
MMLU fraction10.89210.938

Files in This Repository

FileDescription
unlearned_checkpoints.jsonCheckpoint metadata & hyperparameters
evaluation/evaluation_summary.jsonFull evaluation payload (train/test/relearning)
evaluation/score_comparison.csvBaseline vs. unlearned comparison table