Arkhiveus/unaligner1K_DPO
DPO only version of unaligner1K A consolidated and cleaned dataset created from toxic-dpo-v0.2, orthogonal-activation-steering-TOXIC, ToxicQAFinal. The datasets were sorted using Llama-Guard-2 and then randomly sampled. New rejections were generated by Llama-3-8B-Instruct, while new chosen answers for OAS-Toxic and ToxicQA were generated with Nous-Hermes-2-Yi-34B. No of rows from each dataset: OAS-Toxic : 311 ToxicDPO : 478 ToxicQA : 211 Harm occurrence: S1 : 62, S10 : 20, S11 :… See the full description on the dataset page: https://huggingface.co/datasets/Arkhiveus/unaligner1K_DPO.
Conversations for this repository live on Hugging Face.
CoolFace shows imported repositories read-only. Posting into someone else’s repository from here would need an authorised integration and the account holder’s consent, so the link goes to the source instead.
Open discussions on Hugging Face