datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
dolmino-mix-1124
DOLMino dataset mix for OLMo2 stage 2 annealing training.
Mixture of high-quality data used for the second stage of OLMo2 training.
Source Sizes
Name
Category
Tokens
Bytes (uncompressed)
Documents
License
DCLM
HQ Web Pages
752B
4.56TB
606M
CC-BY-4.0
Flan
HQ Web Pages
17.0B
98.2GB
57.3M
ODC-BY
Pes2o
STEM Papers
58.6B
413GB
38.8M
ODC-BY
Wiki
Encyclopedic
3.7B
16.2GB
6.17M
ODC-BY
StackExchange
CodeText
1.26B
7.72GB
2.48M
CC-BY-SA-{2.5, 3.0, 4.0}… See the full description on the dataset page: https://huggingface.co/datasets/allenai/dolmino-mix-1124.ChatGPT-Jailbreak-Prompts
Dataset Card for Dataset Name
Name
ChatGPT Jailbreak Prompts
Dataset Summary
ChatGPT Jailbreak Prompts is a complete collection of jailbreak related prompts for ChatGPT. This dataset is intended to provide a valuable resource for understanding and generating text in the context of jailbreaking in ChatGPT.
Languages
[English]
flores_101One of the biggest challenges hindering progress in low-resource and multilingual machine translation is the
lack of good evaluation benchmarks. Current evaluation benchmarks either lack good coverage of low-resource
languages, consider only restricted domains, or are low quality because they are constructed using
semi-automatic procedures. In this work, we introduce the FLORES evaluation benchmark, consisting of 3001
sentences extracted from English Wikipedia and covering a variety of different topics and domains.
These sentences have been translated in 101 languages by professional translators through a carefully
controlled process. The resulting dataset enables better assessment of model quality on the long tail of
low-resource languages, including the evaluation of many-to-many multilingual translation systems, as all
translations are multilingually aligned. By publicly releasing such a high-quality and high-coverage dataset,
we hope to foster progress in the machine translation community and beyond.po_qwen14b_tabular_data
BoLT Prompt Optimization — Tabular Dataset
For prompt optimization tasks in BoLT, an accessible benchmark for black-box optimization on LLM tasks.
Dataset Description
The dataset covers 5,014 evaluated instructions. Each row is a candidate system-prompt instruction paired with its empirically measured MATH-500 (4-shot, non-thinking mode) scores.
Evaluation details:
Model: Qwen/Qwen3-14B
Task: minerva_math500 (4-shot) (from lm-eval library)
System prompt:… See the full description on the dataset page: https://huggingface.co/datasets/chewwt/po_qwen14b_tabular_data.security-auditsA collection of agent traces generated with Swival (not Claude Code, despite what the HF interface currently shows), an agent designed for open-source models.
These traces focus on security audits of opensource software.
Sharing traces with Swival
Swival can export full conversation traces with --trace-dir, which writes one <session_id>.jsonl file per session:
swival "Fix the login bug" --trace-dir traces/
Those JSONL files use Swival's Claude Code compatible trace export, and… See the full description on the dataset page: https://huggingface.co/datasets/jedisct1/security-audits.soc-builder-rtl-v1
SoC Builder RTL Dataset — v1 (Experiment Release)
A reproducible, machine-generated corpus of synthesizable System-on-Chip (SoC) RTL designs for machine learning on hardware: RTL representation learning today, and — as the corpus grows — netlist, timing, and placement prediction. Every design is a complete, hierarchical, lint-clean Verilog SoC assembled from real open-source IP — RISC-V CPU cores, network-on-chip (NoC) interconnects, accelerators, peripherals, memories and… See the full description on the dataset page: https://huggingface.co/datasets/hasankursun/soc-builder-rtl-v1.helaxai_data_pluse
🥞 The Stack v3
What is it?
What is being released
How to download and use it
Dataset statistics
Dataset structure
Dataset creation
Considerations for using the data
Additional information
What is it?
The Stack v3 is the largest, most up-to-date open dataset of source code, crawled directly from GitHub and built to pre-train code LLMs with full-repository context. It is the successor to The Stack v2 and, like its predecessor, is released to make the training… See the full description on the dataset page: https://huggingface.co/datasets/nhblk123/helaxai_data_pluse.MatrAIx_Persona_1M
MatrAIx Persona 1M
999,847 personas, each described by 1,290 categorical attributes.
599,847 are derived from real records, 400,000 are synthetic.
10 Zstandard Parquet shards, 4.17 GB.
Read it with pyarrow, not datasets
Attributes are packed: one persona's 1,290 attributes are 645 bytes of 4-bit
codes, low nibble first. datasets cannot open these files at all. Use pyarrow
and decode against persona_codes.schema.json.
import json, pyarrow.parquet as pq
schema =… See the full description on the dataset page: https://huggingface.co/datasets/MatrAIx2026/MatrAIx_Persona_1M.PersonaMem-v1🚨 We have now released PersonaMem-v3 and PersonaMem-v2.
This is the official Huggingface repository of the paper Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scale and the PersonaMem benchmark.
We present PersonaMem, a new LLM personalization benchmark to assess how well language models can infer evolving user profiles and generate personalized responses across task scenarios. PersonaMem emphasizes persona-oriented, multi-session… See the full description on the dataset page: https://huggingface.co/datasets/bowen-upenn/PersonaMem-v1.mimo-claude-code-traces-1k
MIMO Claude Code Traces
MIMO Claude Code Traces is a collection of coding-agent trajectories in a Claude Code-style environment. Each record contains a user coding task, the full multi-turn message trace, available tool schemas, assistant reasoning fields, tool calls, tool outputs, and metadata such as model name, category, duration, cost, token usage, and whether the trace used tools.
The traces were generated with mimo-v2.5-pro, MiMo's most capable model at the time of… See the full description on the dataset page: https://huggingface.co/datasets/choucsan/mimo-claude-code-traces-1k.flores_101One of the biggest challenges hindering progress in low-resource and multilingual machine translation is the
lack of good evaluation benchmarks. Current evaluation benchmarks either lack good coverage of low-resource
languages, consider only restricted domains, or are low quality because they are constructed using
semi-automatic procedures. In this work, we introduce the FLORES evaluation benchmark, consisting of 3001
sentences extracted from English Wikipedia and covering a variety of different topics and domains.
These sentences have been translated in 101 languages by professional translators through a carefully
controlled process. The resulting dataset enables better assessment of model quality on the long tail of
low-resource languages, including the evaluation of many-to-many multilingual translation systems, as all
translations are multilingually aligned. By publicly releasing such a high-quality and high-coverage dataset,
we hope to foster progress in the machine translation community and beyond.Magpie-Qwen2.5-Pro-1M-v0.1
Project Web: https://magpie-align.github.io/
Arxiv Technical Report: https://arxiv.org/abs/2406.08464
Codes: https://github.com/magpie-align/magpie
Abstract
Click Here
High-quality instruction data is critical for aligning large language models (LLMs). Although some models, such as Llama-3-Instruct, have open weights, their alignment data remain private, which hinders the democratization of AI. High human labor costs and a limited, predefined scope for prompting prevent… See the full description on the dataset page: https://huggingface.co/datasets/Magpie-Align/Magpie-Qwen2.5-Pro-1M-v0.1.flashmini-data-v1
FlashMini data v4 (card)
Deterministic FlashMini training corpus. Canonical documents live in
Parquet+ZSTD shards under shards/; each shard carries a manifest with
sha256, counts, and distributions; the frozen corpus identity is
corpus_fingerprint_sha256.
Sources and redistribution: each source carries one of mirror_allowed,
recipe_only, gated_recipe_only, review_required, generated_owned
(fail-closed; see registry/sources.yaml + source_snapshot.lock.json).
Content shards are… See the full description on the dataset page: https://huggingface.co/datasets/mjaso/flashmini-data-v1.10k_prompts_ranked
Dataset Card for 10k_prompts_ranked
10k_prompts_ranked is a dataset of prompts with quality rankings created by 314 members of the open-source ML community using Argilla, an open-source tool to label data. The prompts in this dataset include both synthetic and human-generated prompts sourced from a variety of heavily used datasets that include prompts.
The dataset contains 10,331 examples and can be used for training and evaluating language models on prompt ranking tasks. The… See the full description on the dataset page: https://huggingface.co/datasets/data-is-better-together/10k_prompts_ranked.PKU-SafeRLHF-10K
Paper
You can find more information in our paper.
Dataset Paper: https://arxiv.org/abs/2307.04657
hacker-news
Hacker News - Complete Archive
Every Hacker News item since 2006, live-updated every 5 minutes
What is it?
This dataset contains the complete Hacker News archive: every story, comment, Ask HN, Show HN, job posting, and poll ever submitted to the site. Hacker News is one of the longest-running and most influential technology communities on the internet, operated by Y Combinator since 2007. It has become the de facto gathering place for founders, engineers, researchers… See the full description on the dataset page: https://huggingface.co/datasets/whiskey1983/hacker-news.corpus-1T-manifest
SPP Corpus 1T Manifest
The selection manifest for the ~1.0T-token pretraining corpus used in
Synthetic Persona Pretraining (SPP): Alignment from Token Zero.
The corpus is a seeded subsample of allenai/dolma3_mix-6T.
Rather than redistribute ~2.6 TB of text that is already public, this dataset
publishes the selection decisions keyed by upstream document id, so the corpus
can be reconstructed exactly by replaying against upstream.
📄 Reflections + text for the annotated half:… See the full description on the dataset page: https://huggingface.co/datasets/dlab-spp/corpus-1T-manifest.repobench_python_v1.1
RepoBench v1.1 (Python)
Introduction
This dataset presents the Python portion of RepoBench v1.1 (ICLR 2024). The data encompasses a collection from GitHub, spanning the period from October 6th to December 31st, 2023. With a commitment to data integrity, we've implemented a deduplication process based on file content against the Stack v2 dataset (coming soon), aiming to mitigate data leakage and memorization concerns.
Resources and Links
Paper
GitHub… See the full description on the dataset page: https://huggingface.co/datasets/tianyang/repobench_python_v1.1.stack-2021-12-01
ReasonStack-Prime
A highly normalized, streaming-optimized Stack Exchange corpus engineered for LLM reasoning and instruction tuning.
CC BY-SA 4.0
~1M Rows
176 Parquet Shards
21 SE Sites
1. Executive Summary
ReasonStack-Prime is a large-scale, meticulously curated text dataset derived from the official Archive.org Stack Exchange data dump (Version 2021-12-07). Unlike raw XML dumps or poorly cleaned JSON exports, this dataset… See the full description on the dataset page: https://huggingface.co/datasets/AdhyanshVerma/stack-2021-12-01.SWE-Rebench-Tasks-Clean
SWE-Rebench-Tasks-Clean
1,317 verified-solvable, contamination-controlled software-engineering tasks for terminal-agent RL training.
Adapted from nebius/SWE-rebench-V2 (real GitHub issue → PR tasks with executable test contracts) into the TerminalWorld task format. Companion dataset to Fzz1/SWE-Smith-Seeds-Clean, same layout.
Every task is a directory containing:
file
content
instruction.md
the issue text the agent sees (plus linked issue discussion where available)… See the full description on the dataset page: https://huggingface.co/datasets/Fzz1/SWE-Rebench-Tasks-Clean.sutra-1B
Sutra 1B Pretraining Dataset
A high-quality pedagogical dataset designed for LLM pretraining, containing 948,709 educational entries totaling over 1 billion tokens.
Dataset Description
This dataset was generated using the Sutra framework, which creates structured educational content optimized for language model pretraining. Each entry is designed to maximize learning efficiency through:
Clear pedagogical structure: Content follows proven educational patterns
Cross-domain… See the full description on the dataset page: https://huggingface.co/datasets/codelion/sutra-1B.gdb13
GDB13
Info
dataset com aproximadamente 2 bilhões de moleculas de 13 atomos com 7 por heuristica com rdkit
usado para testes de geração condicional baseada nas propriedades ou predição destas.
Foi retirado as informações de posição atomica (grafos) e mantido apenas smiles como informação estrutural
Colunas
smiles: smiles presente no dataset original
canonsmiles: smiles canonizado no rdkit
isSmilesEqual: se smiles riginal é canonico
scaffold: scaffold baseado… See the full description on the dataset page: https://huggingface.co/datasets/raphavlas/gdb13.Pluto-Nano-1.0-Pretrain-v2
ASTRAI Pluto Nano 1.0 — Pretrain Mix (v2)
Curated multilingual pretraining corpus (~50 GB parquet, ~12 B tokens after tokenization) used for ASTRAI Pluto Nano 1.0, a 1 B-total / 50 M-active MoE model with 64 k vocabulary and 5 target languages (EN, PT, ES, ZH, HI).
v2 additions vs v1: OpenThoughts3 (CoT reasoning), openstax textbooks + peS2o (science), and reweighting for better balance. NOTE: factsense (openbmb) was used at training time but is not redistributed here due to its… See the full description on the dataset page: https://huggingface.co/datasets/ASTRAI-labs/Pluto-Nano-1.0-Pretrain-v2.cc100-documents
cc100-documents
This dataset is a restructured version of the CC-100 (statmt/cc100) dataset.
In the original dataset, each instance corresponds to a single paragraph (or a document boundary).
In this version, the data has been reformed so that each instance corresponds to a single, complete document.
This document-level structure makes it more convenient for processing using the map() and filter() methods in the Hugging Face Datasets library.
Languages
The following… See the full description on the dataset page: https://huggingface.co/datasets/singletongue/cc100-documents.KoHRM-Text-1.4B-prepared-data
KoHRM-Text-1.4B Prepared Data
This dataset repository contains prepared HRM-Text V1Dataset artifacts for KoHRM-Text-1.4B.
The data is intended for continued pretraining and staged training with the project code at:
https://github.com/LLM-OS-Models/KoHRM-text
https://huggingface.co/LLM-OS-Models/KoHRM-Text-1.4B
https://huggingface.co/LLM-OS-Models/HRM-Text-Ko-Terminal-Tokenizer-131K
The upstream architecture and training method are based on:
Paper:… See the full description on the dataset page: https://huggingface.co/datasets/LLM-OS-Models/KoHRM-Text-1.4B-prepared-data.Audio-Video-Engineering-Agentic-Tasks-1M
Audio/Video Engineering Agentic Tasks (1M)
Abstract
A highly specialized dataset comprising 1,029,459 in-context troubleshooting prompts and execution commands built for the deepest levels of media production. Unlike standard datasets that simulate clean, theoretical instructions, this matrix captures the chaotic, highly-detailed, and conversational reality of professional audio engineers, composers, and video editors mid-session. It is engineered to train multimodal AI… See the full description on the dataset page: https://huggingface.co/datasets/yatin-superintelligence/Audio-Video-Engineering-Agentic-Tasks-1M.Sujet-Finance-Instruct-177k
Sujet Finance Dataset Overview
The Sujet Finance dataset is a comprehensive collection designed for the fine-tuning of Language Learning Models (LLMs) for specialized tasks in the financial sector. It amalgamates data from 18 distinct datasets hosted on HuggingFace, resulting in a rich repository of 177,597 entries. These entries span across seven key financial LLM tasks, making Sujet Finance a versatile tool for developing and enhancing financial applications of AI.… See the full description on the dataset page: https://huggingface.co/datasets/sujet-ai/Sujet-Finance-Instruct-177k.Creative-Professionals-Agentic-Tasks-1M
Creative Professionals Agentic Tasks (1M)
Abstract
A massive-scale, high-fidelity synthetic task dataset comprising 1,070,917 agentic command operations across 36 creative, technical, and engineering software environments. This dataset is engineered exclusively to stress-test, evaluate, and fine-tune multimodal AI agents designed for Agent Environment operation, complex software interaction, and multi-step reasoning within deep software infrastructures.… See the full description on the dataset page: https://huggingface.co/datasets/yatin-superintelligence/Creative-Professionals-Agentic-Tasks-1M.marinfold-exp11-protein-docs-seq
marinfold-exp11-pdocs-seq
Sequence-only derivative of
eczech/marinfold-exp11-protein-docs.
For every row, the document field has been reduced to just the amino-acid sequence
portion: the <begin_sequence> tag followed by the per-residue three-letter tokens
(e.g. <begin_sequence> <MET> <LYS> <ASN> ...). The <contacts-and-distances-v1>
document-type prefix and everything from <begin_statements> onward (contacts and
distances) are removed. The token format is preserved verbatim so… See the full description on the dataset page: https://huggingface.co/datasets/eczech/marinfold-exp11-protein-docs-seq.opengloss-v1.3-definitions
See also OpenGloss v2.1 (2026-09-07): a deeper release of 109,633 of these headwords — sense-level ids, four reading levels, sense-tagged examples with spans, a judged relation graph, and retrieval supervision — published as a 16-dataset family. v1.3 remains the broader headword list.
OpenGloss Dictionary v1.3 (Definition-Level)
Dataset Summary
OpenGloss is a synthetic encyclopedic dictionary and semantic knowledge graph for English
that integrates lexicographic… See the full description on the dataset page: https://huggingface.co/datasets/mjbommar/opengloss-v1.3-definitions.