sonthenguyen/OpenHermes-2.5-Mistral-7B-mt-bench-DPO-recovered

Training hyperparameters LoRA: r=16 loraalpha=16 loradropout=0.05 bias="none" tasktype="CAUSALLM" targetmodules=['kproj', 'gateproj', 'vproj', 'upproj', 'qproj', 'oproj', 'downproj']

Training arguments: autofindbatchsize=True gradientcheckpointing=True learningrate=5e-7 lrschedulertype="cosine" maxsteps=3922 optim="pagedadamw32bit" warmup_steps=100

DPOTrainer: beta=0.1 maxpromptlength=1024 max_length=1536

Arxiv link: https://arxiv.org/abs/2403.02745