datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
Nemotron-CC-HQ-20B
Nemotron-CC-HQ-20B
This Dataset consists of approximately 20B tokens of Nemotron-CC-HQ, consisting of randomly sampled slices from crawls in the range CC-MAIN-2013-20-part-00012 to CC-MAIN-2019-04-part-00007.
For more information about Nemotron-CC check the Paper by Nvidia
Disclaimer:
Derived from Nemotron-CC (Common Crawl). No ownership of underlying content is claimed.
Data may be subject to third-party rights. Use at your own risk and in compliance with… See the full description on the dataset page: https://huggingface.co/datasets/Fredithefish/Nemotron-CC-HQ-20B.LLaDA-Sample-10BT
Dataset: LLaDA-Sample-10BTBase: HuggingFaceFW/fineweb (subset sample-10BT)Purpose: Training LLaDA (Large Language Diffusion Models)
Preprocessing
Tokenizer: GSAI-ML/LLaDA-8B-Instruct
Chunking: Up to 4,096 tokens per chunk (1% of chunks randomly sized between 1–4,096 tokens)
Noisy masking: Applied with noise factor ε = 1×10⁻³
Fields per chunk (PyTorch tensors):
input_ids
noisy_input_ids
mask
t (time scalar)
Statistics
Total chunks: ~2,520,000… See the full description on the dataset page: https://huggingface.co/datasets/FredyRivera-dev/LLaDA-Sample-10BT.LLaDA-Sample-ES
Dataset: LLaDA-Sample-ES
Base: crscardellino/spanish_billion_words
Purpose: Training LLaDA (Large Language Diffusion Models)
Preprocessing
Tokenizer: GSAI-ML/LLaDA-8B-Instruct
Chunking: Up to 4,096 tokens per chunk (1% of chunks randomly sized between 1–4,096 tokens)
Noisy masking: Applied with noise factor ε = 1×10⁻³
Fields per chunk (PyTorch tensors):
input_ids
noisy_input_ids
mask
t (time scalar)
Statistics
Total chunks: ~ 652,089
Shards: 65… See the full description on the dataset page: https://huggingface.co/datasets/FredyRivera-dev/LLaDA-Sample-ES.stable-diffusion-prompts-2.47M
Source
Combined text-only dataset from
poloclub/diffusiondb
Gustavosta/Stable-Diffusion-Prompts
bartman081523/stable-diffusion-discord-prompts
FredZhang7/krea-ai-prompts
For preprocessing methods, please see Fast GPT2 PromptGen.
Python
Download and save the dataset to all_prompts.txt locally.
pip install datasets
import datasets
dataset = datasets.load_dataset("FredZhang7/stable-diffusion-prompts-2.47M")
train = dataset["train"]
prompts = train["text"]
with… See the full description on the dataset page: https://huggingface.co/datasets/FredZhang7/stable-diffusion-prompts-2.47M.LuxInstruct
LuxInstruct
Dataset Summary
LuxInstruct is the first large-scale cross-lingual instruction tuning dataset for Luxembourgish, introduced in LuxInstruct: A Cross-Lingual Instruction Tuning Dataset For Luxembourgish (Philippy et al., 2025).
It addresses the lack of high-quality instruction–response data for low-resource languages by avoiding direct machine translation into Luxembourgish. Instead, it leverages aligned data from English, French, and German to generate natural… See the full description on the dataset page: https://huggingface.co/datasets/fredxlpy/LuxInstruct.Kronos-Dataset
Kronos-Dataset
It is a collection of various datasets to expand the capabilities of reasoning models in agent tasks, medical reasoning, multilingual thinking, and writing. All of these are unified in a single format:
[
{
"from": "system",
"value": "You are a medical AI assistant with advanced reasoning capabilities. Provide detailed, step-by-step analysis for medical questions."
},
{
"from": "human",
"value": "Given the symptoms of sudden weakness in the left arm and leg, recent… See the full description on the dataset page: https://huggingface.co/datasets/FredyRivera-dev/Kronos-Dataset.SWE-smith-trajectories
SWE-smith Trajectories
Code
•
Paper
•
Site
This dataset contains the 5017 trajectories we fine-tuned Qwen 2.5 Coder Instruct on, leading to
SWE-agent-LM-32B, a coding LM agent that
achieve 40.2% on SWE-bench Verified (no verifiers or multiple rollouts, just 1 attempt per instance).
Trajectories were generated by running SWE-agent + Claude 3.7 Sonnet on task instances from
the SWE-smith dataset.
anime-prompts-180KFor more info on data collection and the preprocessing algorithm, please see Fast Anime PromptGen.
80K unique prompts
safebooru_clean: Cleaned prompts with upscore ≥ 8 from the Safebooru API
For disclaimers about the Danbooru data, please see Danbooru Tag Generator.
100K unique prompts (each)
danbooru_raw: Raw prompts with upscore ≥ 3 from Danbooru API
danbooru_clean: Cleaned prompts with upscore ≥ 3 from Danbooru API
Python
Download and save the dataset… See the full description on the dataset page: https://huggingface.co/datasets/FredZhang7/anime-prompts-180K.openassistant-guanaco-unfiltered
Guanaco-Unfiltered
Any language other than English, German, French, or Spanish has been removed.
Refusals of assistance have been removed.
The identification as OpenAssistant has been removed.
Version 2 is out
Identification as OpenAssistant is now fully removed
other improvements
reap-agent-code
reap-agent-code
Dataset Summary
reap-agent-code is a REAP-style mixed dataset for training LLM coding agents.
It is optimized for agentic coding behavior: writing code, debugging, and tool use.
Each row is JSONL with the schema:
{"text": "..."}
Dataset Composition
Source
Ratio
Count
Signal
evol
45%
9 000
Instruction-to-code
swe
25%
5 000
Bug-fix / problem-solving
xlam
30%
6 000Tool / function calling
Total: 20 000 unique deduplicated… See the full description on the dataset page: https://huggingface.co/datasets/freddm/reap-agent-code.evol-codealpaca-v1
Evolved codealpaca
Updates:
2023/08/26 - Filtered results now only contain pure english instruction and removed any mentioned of trained by OAI response
Median sequence length : 471
We employed a methodology similar to that of WizardCoder, with the exception that ours is open-source. We used the gpt-4-0314 and gpt-4-0613 models to augment and answer each response, with the bulk of generation handled by gpt-4-0314.
The aim of this dataset is twofold: firstly, to facilitate the… See the full description on the dataset page: https://huggingface.co/datasets/freddm/evol-codealpaca-v1.Material_Selection_EvalA benchmark designed to facilitate evaluation and modify the behavior of a foundation model through different existing techniques in the context of material selection for conceptual design.
The data is collected by conducting a survey of experts in the field of material selection. The same questions mentioned in keyquestions.csv are asked to experts.
This can be used to evaluate a Language model performance and its spread compared to a human evaluation.
To get into a more detailed explanation… See the full description on the dataset page: https://huggingface.co/datasets/Frederick001/Material_Selection_Eval.albert-camus-chat-style-chat-dataset
Albert Camus Chat + Style Dataset
This dataset contains the training data used for the Ministral Camus project.
Structure
phase1/style-train.jsonl
Phase 1 style pretraining dataset.
Format: {"text": "..."}
phase2/chat-pairs-corpus-final-clean.jsonl
Phase 2 chat dataset from corpus-derived pairs.
Format: {"messages": [{"role": "system"|"user"|"assistant", "content": "..."}, ...]}
phase2/chat-pairs-light-boost-clean.jsonl
Additional Phase 2 chat pairs for… See the full description on the dataset page: https://huggingface.co/datasets/freddm/albert-camus-chat-style-chat-dataset.