datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
BBQ
A better version of BBQ on Huggingface.
The original dataset didn't put the bias target label along with instances.
Repository for the Bias Benchmark for QA dataset
https://github.com/nyu-mll/BBQ
Authors
Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thompson, Phu Mon Htut, and Samuel R. Bowman.
About BBQ (Paper Abstract)
It is well documented that NLP models learn social biases, but little work has been done on… See the full description on the dataset page: https://huggingface.co/datasets/Elfsong/BBQ.bbq
BBQ Dataset
The Bias Benchmark for Question Answering (BBQ) dataset evaluates social biases in language models through question-answering tasks in English.
Dataset Description
This dataset contains questions designed to test for social biases across multiple demographic dimensions. Each question comes in two variants:
Ambiguous (ambig): Questions where the correct answer should be "unknown" due to insufficient information
Disambiguated (disambig): Questions with… See the full description on the dataset page: https://huggingface.co/datasets/HiTZ/bbq.bbq-sampled-500-each
BBQ Subset for Dike LeaderBoard
We sample 500 examples per bias types from the BBQ dataset to enable fast LLM evaluation for the Dike LeaderBoard.
To evaluate the model on this subset, use the following code:
git clone --depth 1 https://github.com/upunaprosk/lm-evaluation-harness
cd lm-evaluation-harness
pip install -e .
MODEL_NAME=... # meta-llama/Llama-3-8B
lm_eval --model hf \
--model_args pretrained=$MODEL_NAME \
--tasks bbq \
--device cuda:0 \
--batch_size 16… See the full description on the dataset page: https://huggingface.co/datasets/iproskurina/bbq-sampled-500-each.bbq-sampled-100-eachbbq_grillThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/ttokttokttok/bbq_grill.BiasFreeBench-BBQ
BiasFreeBench-BBQ
This is the BBQ dataset used in BiasFreeBench (ICLR 2026). It's from the ambiguous part of BBQ dataset. We extract biased and anti-biased answers. We also provide the outputs for Llama-3.1-8B-Instruct evaluation in 'dialogue'.
bbq_tray_to_grillThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/ttokttokttok/bbq_tray_to_grill.bbq_grill_flipThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/ttokttokttok/bbq_grill_flip.bbq_swappedbbq_roberta_large_race_custom_loss_our_datasetbbq_deberta_v3_large_custom_dataset_custom_headbbq_roberta_large_race_custom_loss_lamda_07_predictionsbbq-glm-4.6bbq_deberta_v3_large_race_custom_loss_less_adapter_categories_predictionsall-bbqBBQ_smolTest: 1800 rows, 200/category, 100 neg / 100 nonneg, stereotyped always at ans0 Few-shot: 900 rows, 25 per (category, polarity, label), label 450/450. No template appears in both splits.
@misc{parrish2022bbqhandbuiltbiasbenchmark,
title={BBQ: A Hand-Built Bias Benchmark for Question Answering},
author={Alicia Parrish and Angelica Chen and Nikita Nangia and Vishakh Padmakumar and Jason Phang and Jana Thompson and Phu Mon Htut and Samuel R. Bowman},
year={2022}… See the full description on the dataset page: https://huggingface.co/datasets/elidek-themis/BBQ_smol.bbq-agebbq_deberta_v3_large_race_custom_loss_less_data_predictionsbbq-nationalitybbq_roberta_large_race_custom_loss_lamda_14_predictionsbbq_deberta_v3_large_race_custom_loss_race_format_predictionsbbq_deberta_v3_large_race_finetuned_predictionsbbq_promptsbbq_roberta_large_race_custom_loss_predictionsbbq_bert_base_cased_race_custom_loss_predictionsbbq-gemini-2.5-probbq_deberta_v3_large_race_custom_loss_custom_datasetbbq_deberta_v3_large_race_custom_loss_lamda_07_predictionsbbq-qwen3-235b-a22b-2507BBQdata source: https://github.com/nyu-mll/BBQ
4 category, each contain 4 topic disjoint with https://huggingface.co/datasets/xxizhouu/social_bias, each topic contains one
ambiguois context+negative question
disambiguated context+negative question
ambiguois context+positive question
disambiguated context+positive question)