datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
JSONSchemaBench
JSONSchemaBench
JSONSchemaBench is a benchmark of real-world JSON schemas designed to evaluate structured output generation for Large Language Models (LLMs). It contains approximately 10,000 JSON schemas, capturing diverse constraints and complexities.
import datasets
from datasets import load_dataset
def main():
# Inspect the available subsets of the datasetall_subsets = datasets.get_dataset_config_names("epfl-dlab/JSONSchemaBench")
print("Available subsets:"… See the full description on the dataset page: https://huggingface.co/datasets/epfl-dlab/JSONSchemaBench.measuring-hate-speech
Dataset card for Measuring Hate Speech
This is a public release of the dataset described in Kennedy et al. (2020) and Sachdeva et al. (2022), consisting of 39,565 comments annotated by 7,912 annotators, for 135,556 combined rows. The primary outcome variable is the "hate speech score" but the 10 constituent ordinal labels (sentiment, (dis)respect, insult, humiliation, inferior status, violence, dehumanization, genocide, attack/defense, hate speech benchmark) can also be treated as… See the full description on the dataset page: https://huggingface.co/datasets/ucberkeley-dlab/measuring-hate-speech.dlgenai-nppe-datasetcodetrans-dlEmploying the MTEB evaluation framework's dataset version, utilize the code below for assessment:
import mteb
import logging
from sentence_transformers import SentenceTransformer
from mteb import MTEB
logger = logging.getLogger(__name__)
model_name = 'intfloat/e5-base-v2'
model = SentenceTransformer(model_name)
tasks = mteb.get_tasks(
tasks=[
"AppsRetrieval",
"CodeFeedbackMT",
"CodeFeedbackST",
"CodeTransOceanContest",
"CodeTransOceanDL"… See the full description on the dataset page: https://huggingface.co/datasets/CoIR-Retrieval/codetrans-dl.corpus-1T-manifest
SPP Corpus 1T Manifest
The selection manifest for the ~1.0T-token pretraining corpus used in
Synthetic Persona Pretraining (SPP): Alignment from Token Zero.
The corpus is a seeded subsample of allenai/dolma3_mix-6T.
Rather than redistribute ~2.6 TB of text that is already public, this dataset
publishes the selection decisions keyed by upstream document id, so the corpus
can be reconstructed exactly by replaying against upstream.
📄 Reflections + text for the annotated half:… See the full description on the dataset page: https://huggingface.co/datasets/dlab-spp/corpus-1T-manifest.reflection-50m
SPP Reflection 50M
The 51.4M-document reflection set from Synthetic Persona Pretraining (SPP):
Alignment from Token Zero — the production half-corpus run, and the dataset the
released models were actually trained on.
🔬 Small sample (same format): dlab-spp/reflection-sample-2k
📉 Earlier 10M run: dlab-spp/reflection-10m
🧾 Safety scores for the full 1T corpus: dlab-spp/safety-classifications
Each row pairs a source document with two generated constitution reflections — a… See the full description on the dataset page: https://huggingface.co/datasets/dlab-spp/reflection-50m.DLR-Web
DLR-Web: Multidisciplinary Reasoning Dataset from Web Corpus [Project Page]
This repository releases the Design-Logic-Reasoning-Web (DLR-Web) dataset from the paper DESIGNER: Design-Logic-Guided Multidisciplinary Data Synthesis for LLM Reasoning (ICLR 2026).
Field definitions
original_document: web-sourced raw document text, further filtered from FineFineWeb; thanks to the FineFineWeb authors and maintainers for providing this resource
design_logic: Design Logic in… See the full description on the dataset page: https://huggingface.co/datasets/Attention1115/DLR-Web.safety-classifications
Safety Annotations for dolma3_mix
Safety score annotations for a 20K-shard subset of allenai/dolma3_mix-6T using
locuslab/safety-classifier_gte-large-en-v1.5.
Schema
Column
Type
Description
id
string
Row identifier (matches source dataset)
safety_score
int8
Argmax safety class (0-5)
safety_probs
list[float32]
Full 6-class probability distribution
Safety scale
Score
Label
Count
Percentage
0
safe
302,972,734
77.39%
1… See the full description on the dataset page: https://huggingface.co/datasets/dlab-spp/safety-classifications.dlgenai-nppe2-datasetdlr_edan_shared_control_lerobotThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.0",
"robot_type": "dlr_edan",
"total_episodes": 104,
"total_frames": 8928,
"total_tasks": 10,
"total_videos": 104,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 5,
"splits": {
"train": "0:104"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/IPEC-COMMUNITY/dlr_edan_shared_control_lerobot.dlr_edan_shared_controlThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.0",
"robot_type": "unknown",
"total_episodes": 104,
"total_frames": 8928,
"total_tasks": 14,
"total_videos": 104,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 5,
"splits": {
"train": "0:104"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/lerobot/dlr_edan_shared_control.dlr_sara_grid_clampThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.0",
"robot_type": "unknown",
"total_episodes": 107,
"total_frames": 7622,
"total_tasks": 1,
"total_videos": 107,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 5,
"splits": {
"train": "0:107"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/lerobot/dlr_sara_grid_clamp.lichess_2025-01_30Mreflection-10m
SPP Reflection 10M
The full ~10M-document reflection set from Synthetic Persona Pretraining (SPP):
Alignment from Token Zero.
📝 Read the post: Synthetic Persona Pretraining: Alignment from Token Zero
🔬 Small sample (same format): dlab-spp/reflection-sample-2k — a 2,000-row sample drawn from this set, for quick inspection.
Each row pairs a pretraining document with a synthetic, value-laden reflection
generated for it: a short first-person (and third-person) moral reflection… See the full description on the dataset page: https://huggingface.co/datasets/dlab-spp/reflection-10m.dlr_sara_pourThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.0",
"robot_type": "unknown",
"total_episodes": 100,
"total_frames": 12971,
"total_tasks": 1,
"total_videos": 100,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 5,
"splits": {
"train": "0:100"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/lerobot/dlr_sara_pour.codetrans-dl-qrels
Dataset Card for "codetrans-dl-qrels"
More Information needed
codetrans-dl-queries-corpusEmploying the CoIR evaluation framework's dataset version, utilize the code below for assessment:
import coir
from coir.data_loader import get_tasks
from coir.evaluation import COIR
from coir.models import YourCustomDEModel
model_name = "intfloat/e5-base-v2"
# Load the model
model = YourCustomDEModel(model_name=model_name)
# Get tasks
#all task ["codetrans-dl","stackoverflow-qa","apps","codefeedback-mt","codefeedback-st","codetrans-contest","synthetic-
# text2sql","cosqa","codesearchnet"… See the full description on the dataset page: https://huggingface.co/datasets/CoIR-Retrieval/codetrans-dl-queries-corpus.sen1floods11-preprocessed-dltcga-dlbc-tabular-open
TCGA-DLBC — Tabular (Open Access)
Open-access TCGA-DLBC data from the NCI Genomic Data Commons, reshaped into one table per GDC data_type. Clinical, biospecimen and every open molecular modality for this cohort, in one place, queryable without downloading a single .tar or parsing a single TSV.
GDC data release: Data Release 46.0 - August 10, 2026
Built: 2026-09-12 03:53:50 UTC
Scope: one TCGA project — see [the family][repo] for the others
from datasets import load_dataset… See the full description on the dataset page: https://huggingface.co/datasets/gabrielaltay/tcga-dlbc-tabular-open.proj-dllm-sftDLC_512x256interaction_protocol
Interaction Protocol Shapes Moral Judgment in Multi-Agent Debate
This repository contains the processed experimental datasets used in:
Interaction Protocol Shapes Moral Judgment in Multi-Agent Debate. Pratik S. Sachdeva and Tom van Nuenen. COLM 2026.
Dataset contents
The experiments/ directory contains Parquet datasets used to reproduce the
figures and analyses in the paper. It includes:
synchronous head-to-head debates;
round-robin head-to-head debates;… See the full description on the dataset page: https://huggingface.co/datasets/ucberkeley-dlab/interaction_protocol.zip2zip-1Bdllm-benchdllm-img-edit-vq-cacheDL-SFT-Datasetdual_so101This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": "dual_so101_follower",
"total_episodes": 2,
"total_frames": 582,
"total_tasks": 1,
"total_videos": 6,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 30,
"splits": {
"train": "0:2"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/dleon23/dual_so101.DLT-Tweets
DLT-Tweets
[Paper] •
[Code]
Dataset Description
Dataset Summary
DLT-Tweets is a large-scale corpus of social media posts related to Distributed Ledger Technology (DLT). This dataset is part of the larger DLT-Corpus collection, designed to support NLP research, social computing studies, and public discourse analysis in the DLT domain. It was introduced in the paper DLT-Corpus: A Large-Scale Text Collection for the Distributed Ledger Technology Domain.… See the full description on the dataset page: https://huggingface.co/datasets/ExponentialScience/DLT-Tweets.dl-npppe2-datasettop_tagging
Dataset Card for Top Quark Tagging
Dataset Summary
Top Quark Tagging is a dataset of Monte Carlo simulated events produced by proton-proton collisions at the Large Hadron Collider. The top-quark signal and mixed quark-gluon background jets are produced with Pythia8 with its default tune for a center-of-mass energy of 14 TeV. Multiple interactions and pile-up are ignored. The leading 200 jet constituent four-momenta (E,px,py,pz) (E, p_x, p_y, p_z) (E,px,py,pz)are stored… See the full description on the dataset page: https://huggingface.co/datasets/dl4phys/top_tagging.