datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
BBQ
BBQ
Repository for the Bias Benchmark for QA dataset.
https://github.com/nyu-mll/BBQ
Authors: Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thompson, Phu Mon Htut, and Samuel R. Bowman.
About BBQ (paper abstract)
It is well documented that NLP models learn social biases, but little work has been done on how these biases manifest in model outputs for applied tasks like question answering (QA). We introduce the Bias Benchmark for QA… See the full description on the dataset page: https://huggingface.co/datasets/walledai/BBQ.BBQ
A better version of BBQ on Huggingface.
The original dataset didn't put the bias target label along with instances.
Repository for the Bias Benchmark for QA dataset
https://github.com/nyu-mll/BBQ
Authors
Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thompson, Phu Mon Htut, and Samuel R. Bowman.
About BBQ (Paper Abstract)
It is well documented that NLP models learn social biases, but little work has been done on… See the full description on the dataset page: https://huggingface.co/datasets/Elfsong/BBQ.bbq_helmBBQ-V
BBQ-V: Benchmarking Visual Stereotype Bias in Large Multimodal Models
⚠️ Content warning: This dataset contains contexts and questions that surface
harmful social stereotypes. It is intended solely for measuring and mitigating bias
in AI systems.
Summary
Stereotype biases in Large Multimodal Models (LMMs) perpetuate harmful societal prejudices, undermining the fairness and equity of AI applications. As LMMs grow increasingly influential, addressing and… See the full description on the dataset page: https://huggingface.co/datasets/ucf-crcv/BBQ-V.bbq
BBQ Dataset
The Bias Benchmark for Question Answering (BBQ) dataset evaluates social biases in language models through question-answering tasks in English.
Dataset Description
This dataset contains questions designed to test for social biases across multiple demographic dimensions. Each question comes in two variants:
Ambiguous (ambig): Questions where the correct answer should be "unknown" due to insufficient information
Disambiguated (disambig): Questions with… See the full description on the dataset page: https://huggingface.co/datasets/HiTZ/bbq.bbq-sampled-500-each
BBQ Subset for Dike LeaderBoard
We sample 500 examples per bias types from the BBQ dataset to enable fast LLM evaluation for the Dike LeaderBoard.
To evaluate the model on this subset, use the following code:
git clone --depth 1 https://github.com/upunaprosk/lm-evaluation-harness
cd lm-evaluation-harness
pip install -e .
MODEL_NAME=... # meta-llama/Llama-3-8B
lm_eval --model hf \
--model_args pretrained=$MODEL_NAME \
--tasks bbq \
--device cuda:0 \
--batch_size 16… See the full description on the dataset page: https://huggingface.co/datasets/iproskurina/bbq-sampled-500-each.bbq-sampled-100-eachBBQ_DPOqa_BBQ_trans_gender
Dataset Card for "qa_BBQ_trans_gender"
More Information needed
bbq_grillThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/ttokttokttok/bbq_grill.cnn_dollybricks_platypus_bbq_2_0
Dataset Card for "cnn_dollybricks_platypus_bbq_2_0"
More Information needed
qa_BBQ_bi_gender
Dataset Card for "qa_BBQ_bi_gender"
More Information needed
BiasFreeBench-BBQ
BiasFreeBench-BBQ
This is the BBQ dataset used in BiasFreeBench (ICLR 2026). It's from the ambiguous part of BBQ dataset. We extract biased and anti-biased answers. We also provide the outputs for Llama-3.1-8B-Instruct evaluation in 'dialogue'.
bbq_cleaned
Dataset Card for "bbq_cleaned"
Get the source data from here: https://huggingface.co/datasets/lighteval/bbq_helm/
And then manually selected.
More Information needed
bbq-ambiguous-unbiased-multi-choicebbq_tray_to_grillThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/ttokttokttok/bbq_tray_to_grill.bbq-gender-bias-free-textbbq_grill_flipThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/ttokttokttok/bbq_grill_flip.bbq-ambiguous-biased-free-textbbq-disambiguated-unbiased-multi-choicebbq_swappedbbq_roberta_large_race_custom_loss_our_datasetbbq_deberta_v3_large_custom_dataset_custom_headbbq-age-bias-free-textbbq-race-unbias-free-textbbq-gender-bias-multi-choicebbq-race-bias-free-textbbq-religion-bias-free-textbbq-race-bias-multi-choicebbq-physical-bias-free-text