Modelpublic

OPTML-Group/SimNPO-MUSE-News-Llama-2-7b

sourceHugging Facemitupdated 2y agoView on Hugging Face

0likes82downloads

Model Card

SimNPO-Unlearned Model on Task "MUSE - News"

Model Details

—Unlearning:
—Task: 🤗datasets/muse-bench/MUSE-News
—Method: SimNPO
—Origin Model: 🤗muse-bench/MUSE-news_target
—Code Base: github.com/OPTML-Group/Unlearn-Simple
—Research Paper: "Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning"

Unlearning Algorithm

This model uses the SimNPO unlearning algorithm with the following optimization objective: $$\ell{SimNPO}(\mathbf{\theta}) = \mathbb{E}{(x, y) \in \mathcal{D}f}\left[-\frac{2}{\beta}\log\sigma\left(-\frac{\beta}{|y|}\log\pi{\mathbf{\theta}}(y|x) - \gamma\right)\right] + \lambda \mathbb{E}{(x, y) \in \mathcal{D}r}[-\log\pi_{\mathbf{\theta}} (y|x)]$$ Unlearning hyper-parameters:

—Learning Rate: 1e-5
—beta: 0.7
—lambda: 1.0
—gamma: 3.0

Loading the Model

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("OPTML-Group/SimNPO-MUSE-News-llama-2-7b", torch_dtype=torch.bfloat16, device_map='auto')

Evaluation Results

	VerbMem Df	KnowMem Df	PrivLeak	KnowMem Dr
Origin	58.29	62.93	-98.71	54.31
Retrain	20.75	33.32	0.00	53.79
NPO	0.00	56.93	56.93	108.91
SimNPO	12.90	47.09	11.90	40.31

Citation

If you use this model in your research, please cite:

@article{fan2024simplicity,
  title={Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning},
  author={Fan, Chongyu and Liu, Jiancheng and Lin, Licong and Jia, Jinghan and Zhang, Ruiqi and Mei, Song and Liu, Sijia},
  journal={arXiv preprint arXiv:2410.07163},
  year={2024}
}

Reporting Issues

Reporting issues with the model: github.com/OPTML-Group/Unlearn-Simple