JoaoBoer/tofu_Llama-3.2-1B-Instruct_forget05_PDU

tofuLlama-3.2-1B-Instructforget05_PDU

open-unlearning/tofu_Llama-3.2-1B-Instruct_full unlearned on the TOFU forget05 split with PDU, trained with the open-unlearning framework. Used as a weight-unlearning baseline / draft model in the Speculative-Decoding-Unlearning project.

Full training config: .hydra/config.yaml. TOFU evaluation outputs: evals/.

Method hyperparameters

gamma: 1.0
alpha: 100
retain_loss_type: NLL
retain_loss_eps: 0.3
primal_dual: True
dual_step_size: 5
dual_update_upon: step
dual_warmup_epochs: 5
loss_names: ['forget_loss', 'retain_loss']

TOFU summary metrics