Arkhiveus/unaligner1K

A consolidated and cleaned dataset created from toxic-dpo-v0.2, orthogonal-activation-steering-TOXIC, ToxicQAFinal. The datasets were sorted using Llama-Guard-2 and then randomly sampled. New rejections were generated by Llama-3-8B-Instruct, while new chosen answers for OAS-Toxic and ToxicQA were generated with Nous-Hermes-2-Yi-34B. Disclaimers and warnings were then manually removed from the chosen answer. No of rows from each dataset: OAS-Toxic : 311 ToxicDPO : 478 ToxicQA : 211 Harm… See the full description on the dataset page: https://huggingface.co/datasets/Arkhiveus/unaligner1K.