darklord1611/LLaDA-8B-Instruct-em-bad-medical-advice-run1

darklord1611/LLaDA-8B-Instruct-em-bad-medical-advice-run1

LoRA adapter for studying Emergent Misalignment in diffusion language models.

Training Details

• —Base model: GSAI-ML/LLaDA-8B-Instruct
• —Dataset: bad-medical-advice
• —Method: LoRA (r=32, alpha=64, target_modules=all-linear)
• —Epochs: 1
• —Seed: 1
• —Learning rate: 1e-5

Usage

from peft import PeftModel
from transformers import AutoModelForMaskedLM

base = AutoModelForMaskedLM.from_pretrained("GSAI-ML/LLaDA-8B-Instruct", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "darklord1611/LLaDA-8B-Instruct-em-bad-medical-advice-run1")
model = model.merge_and_unload()

Project

Part of the em-diffusion research project studying emergent misalignment in diffusion LMs.