datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
PKU-SafeRLHF
Dataset Card for PKU-SafeRLHF
Warning: this dataset contains data that may be offensive or harmful. The data are intended for research purposes, especially research that can make models less harmful. The views expressed in the data do not reflect the views of PKU-Alignment Team or any of its members.
[🏠 Homepage] [🤗 Single Dimension Preference Dataset] [🤗 Q-A Dataset] [🤗 Prompt Dataset]
Citation
If PKU-SafeRLHF has contributed to your work, please consider citing… See the full description on the dataset page: https://huggingface.co/datasets/PKU-Alignment/PKU-SafeRLHF.PKU-SafeRLHF-10K
Paper
You can find more information in our paper.
Dataset Paper: https://arxiv.org/abs/2307.04657
PKU-SafeRLHF-30K
Dataset Card for PKU-SafeRLHF
Warning: this dataset contains data that may be offensive or harmful. The data are intended for research purposes, especially research that can make models less harmful. The views expressed in the data do not reflect the views of PKU-Alignment Team or any of its members.
Dataset Summary
The preference dataset consists of 30k+ expert comparison data. Each entry in this dataset includes two responses to a question, along with safety… See the full description on the dataset page: https://huggingface.co/datasets/PKU-Alignment/PKU-SafeRLHF-30K.PKU-SafeRLHF-QA
Dataset Card for PKU-SafeRLHF-QA
Warning: this dataset contains data that may be offensive or harmful. The data are intended for research purposes, especially research that can make models less harmful. The views expressed in the data do not reflect the views of PKU-Alignment Team or any of its members.
Dataset Summary
This dataset contains 265K Q-A pairs, including all Q-A pairs from PKU-SafeRLHF. You can use sha256 to match corresponding data between two datasets. Each… See the full description on the dataset page: https://huggingface.co/datasets/PKU-Alignment/PKU-SafeRLHF-QA.PKU-SafeRLHF-single-dimension
Dataset Card for PKU-SafeRLHF-single-dimension
Warning: this dataset contains data that may be offensive or harmful. The data are intended for research purposes, especially research that can make models less harmful. The views expressed in the data do not reflect the views of PKU-Alignment Team or any of its members.
Dataset Summary
By annotating Q-A-B pairs in PKU-SafeRLHF with single dimension, this dataset provide 81.1K high quality preference dataset. Specifically… See the full description on the dataset page: https://huggingface.co/datasets/PKU-Alignment/PKU-SafeRLHF-single-dimension.PKU-SafeRLHF-prompt
Dataset Card for PKU-SafeRLHF-prompt
This dataset contains 44.6K unique prompts from PKU-SafeRLHF. 22.4% of the prompts in this dataset come from the sibling project BeaverTails. Additionally, we performed SFT on Llama3-70B using the Alpaca 52K dataset, resulting in Alpaca3-70B. 63.6% and 14.0% of our dataset is generated by Alpaca3-70B and WizardLM-30B-Uncensored, respectively, under the guidance of experts.
Here is the generation pipeline:
Usage
To load our dataset… See the full description on the dataset page: https://huggingface.co/datasets/PKU-Alignment/PKU-SafeRLHF-prompt.PKU-SafeRLHF
Dataset Card for PKU-SafeRLHF
Warning: this dataset contains data that may be offensive or harmful. The data are intended for research purposes, especially research that can make models less harmful. The views expressed in the data do not reflect the views of PKU-Alignment Team or any of its members.
[🏠 Homepage] [🤗 Single Dimension Preference Dataset] [🤗 Q-A Dataset] [🤗 Prompt Dataset]
Citation
If PKU-SafeRLHF has contributed to your work, please consider… See the full description on the dataset page: https://huggingface.co/datasets/Kanika0110/PKU-SafeRLHF.PKU-SafeRLHF-QA
Dataset Card for PKU-SafeRLHF-QA
Warning: this dataset contains data that may be offensive or harmful. The data are intended for research purposes, especially research that can make models less harmful. The views expressed in the data do not reflect the views of PKU-Alignment Team or any of its members.
Dataset Summary
This dataset contains 265K Q-A pairs, including all Q-A pairs from PKU-SafeRLHF. You can use sha256 to match corresponding data between two… See the full description on the dataset page: https://huggingface.co/datasets/Kanika0110/PKU-SafeRLHF-QA.PKU-SafeRLHF-orpo-72kWarning: this dataset contains data that may be offensive or harmful. The data are intended for research purposes, especially research that can make models less harmful.
👇original PKU-SafeRLHF datasets (click 🔗 for more details)
what's the advantage of this train dataset over the original one ?
standard chosen/rejected format of preference datasets : make 'chosen' and 'rejected' according to 'better_response_id'
only one file : merge three train datasets(Alpaca-7B、Alpaca2-7B、Alpaca3-8B)… See the full description on the dataset page: https://huggingface.co/datasets/juneup/PKU-SafeRLHF-orpo-72k.PKU-SafeRLHF-30K
Dataset Card for PKU-SafeRLHF
Warning: this dataset contains data that may be offensive or harmful. The data are intended for research purposes, especially research that can make models less harmful. The views expressed in the data do not reflect the views of PKU-Alignment Team or any of its members.
Dataset Summary
The preference dataset consists of 30k+ expert comparison data. Each entry in this dataset includes two responses to a question, along with safety… See the full description on the dataset page: https://huggingface.co/datasets/Kanika0110/PKU-SafeRLHF-30K.PKU-Alignment_PKU-SafeRLHF-Safer-PreferenceShareGPTPKU-SafeRLHF-ko
Original Dataset: PKU-Alignment/PKU-SafeRLHF
Translation by using maywell/Synatra-7B-v0.3-Translation
Translating in progress...
PKU-SafeRLHF
Dataset Card for PKU-SafeRLHF
Warning: this dataset contains data that may be offensive or harmful. The data are intended for research purposes, especially research that can make models less harmful. The views expressed in the data do not reflect the views of PKU-Alignment Team or any of its members.
[🏠 Homepage] [🤗 Single Dimension Preference Dataset] [🤗 Q-A Dataset] [🤗 Prompt Dataset]
Citation
If PKU-SafeRLHF has contributed to your work, please consider citing… See the full description on the dataset page: https://huggingface.co/datasets/sdzxc321/PKU-SafeRLHF.PKU-Alignment_PKU-SafeRLHF-Better-PreferenceShareGPTPKU-SafeRLHF-30K-Embedding-Rewardimport numpy as np
import json
reward = np.load("./scores.npy")
embedding_list = []
with open("./embeddings.jsonl", "r") as f:
for line in f:
line_data = json.loads(line)
embedding_list.append(line_data)
PKU-Alignment_PKU-SafeRLHF-PreferenceShareGPTOnly kept better_response_id != safer_response_id.
saferlhf_evaluation_datasetPKU-SafeRLHF-RefusalsPKU-SafeRLHF-RLHF
PKU-SafeRLHF-RLHF
A single RLHF-ready reward config derived from
PKU-Alignment/PKU-SafeRLHF.
config
columns
use case
reward
prompt, chosen, rejected, margin
reward-model training
Example counts
split
examples
train
33,334
test
3,688
total
** 37,022**
SFT and DPO projections are trivially derived from this config without
re-downloading:
sft: keep prompt + chosen
dpo: keep prompt + chosen + rejected
Filtering
Three… See the full description on the dataset page: https://huggingface.co/datasets/AIPlans/PKU-SafeRLHF-RLHF.saferlhf-iter1-annotation-gemma-2-2b-sft.jsonlPKU-SafeRLHF-10K
Paper
You can find more information in our paper.
Dataset Paper: https://arxiv.org/abs/2307.04657