CoolFace
Modelpublic

shirasko/llama-3.1-8b-instruct-crisp-ancient-rome

sourceHugging Faceupdated 2mo agoView on Hugging Face
0likes14downloads
Model Card

Unlearned Checkpoint

FieldValue
Unlearning methodCRISP
Base modelmeta-llama/Llama-3.1-8B-Instruct
Target conceptAncient Rome
Checkpoint typeLoRA Adapter
Rank / seed200 / 42
Train eval protocolmc

Unlearning Configuration

Selected hyperparameters (from unlearned_checkpoints.json):

ParameterValue
alpha20
delta_embed0
k_features20
k_features_embed0
layer_hi19
layer_lo5
layer_step2
lora_rank4
lr0.0005
n_tokens_edited0
num_epochs2

Primary Unlearning Metrics (held-out test, MC protocol)

Headline scores used for checkpoint selection:

MetricTrain (after unlearning)**Test (after unlearning)**
Efficacy0.8660.725
Specificity0.6940.82
Harmonic mean0.7710.769
Relearning QA (MC)—0.88

Full Evaluation (baseline → unlearned)

From evaluation/score_comparison.csv:

MetricBaseline (train)After unlearn (train)Baseline (test)**After unlearn (test)**
QA accuracy0.920.340.940.44
QA fraction10.13410.275
SimDom accuracy0.940.680.90.86
SimDom fraction10.62310.938
MMLU accuracy0.620.540.650.541
MMLU fraction10.78410.728

Files in This Repository

FileDescription
unlearned_checkpoints.jsonCheckpoint metadata & hyperparameters
evaluation/evaluation_summary.jsonFull evaluation payload (train/test/relearning)
evaluation/score_comparison.csvBaseline vs. unlearned comparison table