JoaoBoer/tofu_Llama-3.1-8B-Instruct_forget05_RMU

tofuLlama-3.1-8B-Instructforget05_RMU

open-unlearning/tofu_Llama-3.1-8B-Instruct_full unlearned on the TOFU forget05 split with RMU, trained with the open-unlearning framework. Used as a weight-unlearning baseline / draft model in the Speculative-Decoding-Unlearning project.

Full training config: .hydra/config.yaml. TOFU evaluation outputs: evals/.

Method hyperparameters

gamma: 1.0
alpha: 1
retain_loss_type: EMBED_DIFF
steering_coeff: 1
module_regex: model\.layers\.5
trainable_params_regex: ['.*']

TOFU summary metrics