datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
contentsopc-fineweb-code-corpus
OpenCoder Dataset
The OpenCoder dataset is composed of the following datasets:
opc-sft-stage1: the sft data used for opencoder sft-stage1
opc-sft-stage2: the sft data used for opencoder sft-stage2
opc-annealing-corpus: the synthetic data & algorithmic corpus used for opencoder annealing
opc-fineweb-code-corpus: the code-related page recalled from fineweb <-- you are here
opc-fineweb-math-corpus: the math-related page recalled from finewebrefineCode-code-corpus-meta: the meta-data… See the full description on the dataset page: https://huggingface.co/datasets/OpenCoder-LLM/opc-fineweb-code-corpus.Unified-FeedbackCollections of pairwise feedback datasets.
openai/summarize_from_feedback
openai/webgpt_comparisons
Dahoas/instruct-synthetic-prompt-responses
Anthropic/hh-rlhf
lmsys/chatbot_arena_conversations
openbmb/UltraFeedback
argilla/ultrafeedback-binarized-preferences-cleaned
berkeley-nest/Nectar
Codes to reproduce the dataset: jdf-prog/UnifiedFeedback
Dataset formats
{
"id": "...",
"conv_A": [
{
"role": "user",
"content": "...",
},
{
"role": "assistant"… See the full description on the dataset page: https://huggingface.co/datasets/llm-blender/Unified-Feedback.OpenScience
Open Science Dataset
Overview
Open Science is a large-scale, permissively licensed text dataset derived from OpenAlex, containing over 100B (105,390,332,599) words. OpenAlex is an open database of scholarly publications, authors, institutions, and research outputs that serves as a comprehensive source for academic literature.
Key Features
Truly Open: Contains only permissively licensed data suitable for both commercial and non-commercial use
Multilingual… See the full description on the dataset page: https://huggingface.co/datasets/LLMDH/OpenScience.cissp-llmbench
CISSP-LLMBench
otheragent-llm-traces-v2
Exgentic Agent LLM Traces v2 — Agent Chat Only
OpenTelemetry-shaped execution traces for 10,057 agent runs across 6 benchmarks (AppWorld, SWE-bench, BrowseCompPlus, τ²-bench Airline/Retail/Telecom), filtered to the agent under test's chat-only LLM calls. This is the dataset for replay testing, behavioral analysis, or any task where you care about what the benchmarked model actually did — not the eval scaffolding around it.
This v2 release expands upon Exgentic/agent-llm-traces… See the full description on the dataset page: https://huggingface.co/datasets/Exgentic/agent-llm-traces-v2.leaderboard-contents-v2llm-bias-detection
LLM Bias Detection Evaluation Traces
Evaluation data accompanying Navigating the digital spectrum: Assessing
political bias, stability, and downstream fairness in Large Language Models
(arXiv:2609.08637).
Licence scope: CC BY 4.0 covers the authors' original documentation,
templates, selection/arrangement and author-generated tables. It does not
relicense source text or annotations. IBM retains CC BY-SA 3.0; hate-corpus
components retain CC BY 4.0, CC0 or MIT as documented in… See the full description on the dataset page: https://huggingface.co/datasets/nishan-chatterjee/llm-bias-detection.RefineCode-code-corpus-metaThis dataset consists of meta information (including the repository name and file path) of the raw code data from RefineCode. You can collect those files referring to this metadata and reproduce RefineCode!
Note: Currently, we have uploaded the meta data covered by The Stack V2 (About 50% file volume). Due to complex legal considerations, we are unable to provide the complete source code currently. We are working hard to make the remaining part available.
RefineCode is a high-quality… See the full description on the dataset page: https://huggingface.co/datasets/OpenCoder-LLM/RefineCode-code-corpus-meta.LLM-TTA-Augmentation-Logsagent-llm-traces
Multi-Benchmark LLM Agent Traces
A comprehensive dataset of OpenTelemetry traces capturing LLM inference behavior across multiple agent frameworks, benchmarks, and model providers. This dataset enables research into LLM performance analysis, agent behavior patterns, and inference optimization.
Collected by Exgentic - A platform for LLM observability and performance optimization.
Dataset Overview
This dataset contains 1,781 execution traces capturing detailed agent… See the full description on the dataset page: https://huggingface.co/datasets/Exgentic/agent-llm-traces.0399-tv-valid-clean-sft-tokenized-llmjp4-8bWiki-JA-Pair
Wiki-JA-Pair
Wiki-JA-Pair is a dataset of 1M image–text pairs sourced from the Japanese Wikipedia (Wiki-JA).
This dataset is constructed using the May 1, 2025 dump (jawiki-20250501-pages-articles-multistream.xml.bz2).
How to Use
from datasets import load_dataset
ds = load_dataset("llm-jp/Wiki-JA-Pair")
Format
Wiki-JA-Pair includes the following columns:
url: URL of the image
caption: Caption associated with the image
description: Nearby text that… See the full description on the dataset page: https://huggingface.co/datasets/llm-jp/Wiki-JA-Pair.droid_1.0.1hiring-bias-mitigation-responses
Hiring-bias mitigation — model responses
Every response produced in the mitigation study of LLM hiring decisions: 61 runs,
2,689,200 responses, from 5 open-weight models in English and Ukrainian, at
baseline and under each mitigation family (baseline, embedding, prompt, scrub, sft). Each run is one subset.
All released artifacts: the Hiring Bias Mitigation collection.
Training data of the fine-tuned runs: hiring-bias-mitigation-synthetic-data.
Code, configs, full results and… See the full description on the dataset page: https://huggingface.co/datasets/Stereotypes-in-LLMs/hiring-bias-mitigation-responses.llm-jailbreak-prompt-injection-dataset
LLM Jailbreak & Prompt-Injection Dataset
A unified safety dataset combining 30+ public sources for training LLM
guardrails, content moderation classifiers, and response-safety filters.
Schema (orthogonal multi-label, WildGuard-style)
Instead of a single binary is_dangerous, every example carries four
orthogonal labels matching the structure used by AI2 WildGuard, IBM Granite
Guardian, and Azure Prompt Shields:
Column
Type
Description
prompt
str
The user/attack… See the full description on the dataset page: https://huggingface.co/datasets/Necent/llm-jailbreak-prompt-injection-dataset.AlGhafa-Arabic-LLM-Benchmark-Translatedllm-knowledge-collapse
"Epistemic Diversity and Knowledge Collapse in Large Language Models" (Wright et al. 2025)
Authors: Dustin Wright, Sarah Masud, Jared Moore, Srishti Yadav, Maria Antoniak, Peter Ebert Christiensen, Chan Young Park, and Isabelle Augenstein
Contains all 1.6M responses and 70M claims used to measure LLM epistemic diversity in the paper "Epistemic Diversity and Knowledge Collapse in Large Language Models" (Wright et al. 2025)
@article{wright2025epistemicdiversity… See the full description on the dataset page: https://huggingface.co/datasets/dwright37/llm-knowledge-collapse.llm_plan_gen_dataset_accu_t1_t3_t4
[!IMPORTANT]
This is the training dataset for the ICAPS 2025 paper "Chasing Progress, Not Perfection: Revisiting Strategies for End-to-End LLM Plan Generation".
from pathlib import Path
import os
import jsonlines
from copy import deepcopy
from datasets import load_dataset
from icecream import ic
import enum
from enum import IntEnum
from enum import auto
class CONFIG_TYPES(enum.Enum):
# "type_id": ['t0', 'accu-t1', 'accu-t2', 'accu-t3', 'accu-t4', 'accu-t4', 'accu-t1+t4', 'accu-t2+t4']… See the full description on the dataset page: https://huggingface.co/datasets/huangsukai/llm_plan_gen_dataset_accu_t1_t3_t4.llm_plan_gen_dataset_accu_t4
[!IMPORTANT]
This is the training dataset for the ICAPS 2025 paper "Chasing Progress, Not Perfection: Revisiting Strategies for End-to-End LLM Plan Generation".
from pathlib import Path
import os
import jsonlines
from copy import deepcopy
from datasets import load_dataset
from icecream import ic
import enum
from enum import IntEnum
from enum import auto
class CONFIG_TYPES(enum.Enum):
# "type_id": ['t0', 'accu-t1', 'accu-t2', 'accu-t3', 'accu-t4', 'accu-t4', 'accu-t1+t4', 'accu-t2+t4']… See the full description on the dataset page: https://huggingface.co/datasets/huangsukai/llm_plan_gen_dataset_accu_t4.llm-graph-poisoning-data
Generation-Time Poisoning of LLM-Generated Social Networks
This dataset contains synthetic personas, LLM-generated social graphs, cached
text embeddings, and evaluation metrics for clean generation and three
generation-time attack families. All names and profiles are synthetic and do
not represent real people.
Dataset variants
Variant
Nodes
Generator
Graph seeds per condition
Attack rates
p50
50
Qwen3-Max
10
10%, 20%, 30%, 40%, 50%
p200
200… See the full description on the dataset page: https://huggingface.co/datasets/Kevynf/llm-graph-poisoning-data.asap-7-originalspai-ss6-llm-1b-thai-corpus
Thai Medical And Health Corpus
Thai public medical and health web corpus collected for research and LLM dataset
experimentation, with optional imported Thai medical/health datasets from
Hugging Face stored as separate configs.
Public Web Corpus
Config: default
Split: train
Records: 3660 deduplicated articles
Columns: 16
Format: Parquet
Latest collection profile: free_1000
Latest generated at: 2026-06-06T17:41:38.787978+00:00
Source And Method
The… See the full description on the dataset page: https://huggingface.co/datasets/SPAISS6F1/spai-ss6-llm-1b-thai-corpus.opencl-llmperfA collection of datasets and benchmarks for large-scale Performance Modeling with LLMs.
This collection includes these dataset:
github-200K: a first version dataset that contains ~1300 kernel's execution time with input size - global size correlation and imbalanced data.
github-350k: a second version dataset continued from the first that fix the imbalanced data problem.
github-400k: a third version dataset continued from the second that relax the input - global size correlation.
github-600K:… See the full description on the dataset page: https://huggingface.co/datasets/minhkhoi1026/opencl-llmperf.opc-fineweb-math-corpus
OpenCoder Dataset
The OpenCoder dataset is composed of the following datasets:
opc-sft-stage1: the sft data used for opencoder sft-stage1
opc-sft-stage2: the sft data used for opencoder sft-stage2
opc-annealing-corpus: the synthetic data & algorithmic corpus used for opencoder annealing
opc-fineweb-code-corpus: the code-related page recalled from fineweb
opc-fineweb-math-corpus: the math-related page recalled from fineweb <-- you are here
refineCode-code-corpus-meta: the… See the full description on the dataset page: https://huggingface.co/datasets/OpenCoder-LLM/opc-fineweb-math-corpus.llm-cognitive-choice
Do LLMs Choose Like Humans?
Data for Do LLMs Choose Like Humans? Using Cognitive Theory to Evaluate LLM Decision-Making, by Johnathan Sun, Andrei Shleifer, and Yonatan Belinkov.
The dataset contains the product choice trials, model responses, stimuli, and ratings used in the paper. The files follow the layout expected by the analysis code.
The download is about 3.42 GB and includes 2,240,800 recorded model responses across 51 files.
Browse the files in Data Studio. Use the… See the full description on the dataset page: https://huggingface.co/datasets/johnathansun/llm-cognitive-choice.classifier_source
Dataset Card for Lapa High Quality Pretraining Dataset
Dataset Description
Dataset Summary
This dataset is a random sample of both https://huggingface.co/datasets/lapa-llm/pretraining-lower-quality and https://huggingface.co/datasets/lapa-llm/pretraining-high-quality to transfer classifiers from English language to Ukrainian.It was used to transfer the following models from this collection https://huggingface.co/collections/lapa-llm/lapa-v012-pretraining:… See the full description on the dataset page: https://huggingface.co/datasets/lapa-llm/classifier_source.relaion2B-en-research-safe-japanese-translation
relaion2B-en-research-safe-japanese-translation
This dataset is the Japanese translation of the English subset of ReLAION-5B (laion/relaion2B-en-research-safe), translated by gemma-2-9b-it.
We used text2dataset for translating with open-weight LLMs.
By leveraging the fast LLM inference library vLLM, this tool enables the rapid translation of large English datasets into Japanese.
Prompt
The following is the prompt used for translation with Gemma.
You are an… See the full description on the dataset page: https://huggingface.co/datasets/llm-jp/relaion2B-en-research-safe-japanese-translation.asap-8-original