Modelpublic

basab1142/hawkes-nsgo-llama3-ns-dpo-r2

sourceHugging Faceupdated 2mo agoView on Hugging Face

0likes5downloads

Model Card

basab1142/hawkes-nsgo-llama3-ns-dpo-r2

Round 2 LoRA adapter trained with NS-DPO on kC-NSGO dataset.

Field	Value
Base model	`meta-llama/Meta-Llama-3.1-8B-Instruct`
Method	NS-DPO
Regime	`adversarial_ns_dpo`
Round	2
lr	5e-7
Continued from	Round 1 QLoRA checkpoint
Final reward accuracy	nan
LoRA rank	16

Load

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Meta-Llama-3.1-8B-Instruct",
    torch_dtype="bfloat16", device_map="auto"
)
model = PeftModel.from_pretrained(base, "basab1142/hawkes-nsgo-llama3-ns-dpo-r2")
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3.1-8B-Instruct")