datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
general-instruction-augmented-corpora
Instruction Pre-Training: Language Models are Supervised Multitask Learners (EMNLP 2024)
This repo contains the general instruction-augmented corpora (containing 200M instruction-response pairs covering 40+ task categories) used in our paper Instruction Pre-Training: Language Models are Supervised Multitask Learners.
We explore supervised multitask pre-training by proposing Instruction Pre-Training, a framework that scalably augments massive raw corpora with instruction-response… See the full description on the dataset page: https://huggingface.co/datasets/instruction-pretrain/general-instruction-augmented-corpora.python_code_instructions_18k_alpaca
Dataset Card for python_code_instructions_18k_alpaca
The dataset contains problem descriptions and code in python language.
This dataset is taken from sahil2801/code_instructions_120k, which adds a prompt column in alpaca style. Refer to the source here.
CodeFeedback-Filtered-Instruction OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement
[🏠Homepage]
|
[🛠️Code]
OpenCodeInterpreter
OpenCodeInterpreter is a family of open-source code generation systems designed to bridge the gap between large language models and advanced proprietary systems like the GPT-4 Code Interpreter. It significantly advances code generation capabilities by integrating execution and iterative refinement functionalities.
For further information and… See the full description on the dataset page: https://huggingface.co/datasets/m-a-p/CodeFeedback-Filtered-Instruction.molecule_property_instruction
Dataset Card for "molecule_property_instruction"
More Information needed
tulu-3-sft-personas-instruction-following
Dataset Descriptions
This dataset contains 29980 examples and is synthetically created to enhance model's capabilities to follow instructions precisely and to satisfy user constraints. The constraints are borrowed from the taxonomy in IFEval dataset.
To generate diverse instructions, we expand the methodology in Ge et al., 2024 by using personas. More details and exact prompts used to construct the dataset can be found in our paper.
Curated by: Allen Institute for AI
Paper: TBD… See the full description on the dataset page: https://huggingface.co/datasets/allenai/tulu-3-sft-personas-instruction-following.natural-instructionsPreprocessed version of Super-Natural-Instructions from https://github.com/allenai/natural-instructions/tree/master/splits. The same inputs may appear with different outputs, thus to avoid duplicate inputs, you can deduplicate by the id or the inputs field.
Train Tasks:
['task001_quoref_question_generation', 'task002_quoref_answer_generation', 'task022_cosmosqa_passage_inappropriate_binary', 'task023_cosmosqa_question_generation', 'task024_cosmosqa_answer_generation'… See the full description on the dataset page: https://huggingface.co/datasets/Muennighoff/natural-instructions.instruction-datasetThis is the blind eval dataset of high-quality, diverse, human-written instructions with demonstrations. We will be using this for step 3 evaluations in our RLHF pipeline.
instruction_following
Dataset Card for "livebench/instruction_following"
LiveBench is a benchmark for LLMs designed with test set contamination and objective evaluation in mind. It has the following properties:
LiveBench is designed to limit potential contamination by releasing new questions monthly, as well as having questions based on recently-released datasets, arXiv papers, news articles, and IMDb movie synopses.
Each question has verifiable, objective ground-truth answers, allowing hard questions… See the full description on the dataset page: https://huggingface.co/datasets/livebench/instruction_following.Trendyol-Cybersecurity-Instruction-Tuning-Dataset
Trendyol Cybersecurity Defense Instruction-Tuning Dataset (v2.0)
🚀 TL;DR
53,202 meticulously curated system/user/assistant instruction-tuning examples covering 200+ specialized cybersecurity domains. Built by the Trendyol Security Team for training state-of-the-art defensive security AI assistants. Expanded from 21K to 53K rows with comprehensive coverage of modern security challenges including cloud-native threats, AI/ML security, quantum computing risks… See the full description on the dataset page: https://huggingface.co/datasets/Trendyol/Trendyol-Cybersecurity-Instruction-Tuning-Dataset.code_instructions_122k_alpaca_styleramanv-image-vlm-instructionVLA_Instruction_TuningThis repository contains the VLA-IT dataset, a curated 650K-sample Vision-Language-Action Instruction Tuning dataset, and the SimplerEnv-Instruct benchmark. These are presented in the paper InstructVLA: Vision-Language-Action Instruction Tuning from Understanding to Manipulation. The dataset is designed to enable robots to integrate multimodal reasoning with precise action generation, preserving the flexible reasoning of large vision-language models while delivering leading manipulation… See the full description on the dataset page: https://huggingface.co/datasets/ShuaiYang03/VLA_Instruction_Tuning.Nemotron-SFT-Instruction-Following-Chat-v3
Dataset Description:
The Nemotron-Instruction-Following-Chat-v3 dataset is designed to strengthen multi-turn, interactive capabilities, including open-ended chat and precise instruction following.
The chat subset uses human written prompts from sources like lmarena, lmsys, and wildchat as seed prompts. Responses are generated with GLM-5. Multiple responses are sampled from the model and the best response as judged by pairwise comparisons using Qwen3-Nemotron-235B-A22B-GenRM-2603… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/Nemotron-SFT-Instruction-Following-Chat-v3.Nemotron-SFT-Instruction-Following-Chat-v2
Dataset Description:
The Nemotron-Instruction-Following-Chat-v2 dataset is designed to broadly strengthen the model’s interactive capabilities, including open-ended chat and precise instruction following.The dataset is a refreshed version of Nemotron-Instruction-Following-Chat-v1 with synthetic dialogues generated from Kimi-K2-Thinking, GLM-4.6, Qwen3-235B-A22B-Thinking-2507, GPT-OSS-120b, Kimi-K2-Instruct-0905, and Qwen3-235B-A22B-Instruct-2507.
This dataset is ready for commercial… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/Nemotron-SFT-Instruction-Following-Chat-v2.instruction-following-evalhelpful-instructions
Dataset Card for Helpful Instructions
Dataset Summary
Helpful Instructions is a dataset of (instruction, demonstration) pairs that are derived from public datasets. As the name suggests, it focuses on instructions that are "helpful", i.e. the kind of questions or tasks a human user might instruct an AI assistant to perform. You can load the dataset as follows:
from datasets import load_dataset
# Load all subsets
helpful_instructions =… See the full description on the dataset page: https://huggingface.co/datasets/HuggingFaceH4/helpful-instructions.sec-contracts-financial-extraction-instructions
S&P 500 SEC Financial Extraction Instructions
Dataset Summary
7,683 instruction-tuning examples for training LLMs to extract structured financial data from SEC filings. Covers two filing types across S&P 500 companies:
Split
Examples
Filing Type
Description
train
3,430
Exhibit 10 + DEF 14A
Positive examples with validated outputs
corrective
4,253
Exhibit 10 + DEF 14A
Corrective, rescued, and negative examples
Exhibit 10 — Material Contracts (2… See the full description on the dataset page: https://huggingface.co/datasets/TheTokenFactory/sec-contracts-financial-extraction-instructions.train_video_and_instruction
ShareGPTVideo Training Data
All dataset and models can be found at ShareGPTVideo.
Contents:
Train 300k video frames: contains video frames used for SFT and DPO model, which is a subset of total 900k.
ActivityNet 50k + vidal 150k + webvid 100k.
Train 600k video frames: contains the rest 600k frames, the total 900k frames are used for pre-training stage. If you just do finetuning using our video QA, you can just download the 300k above.
900k composition is 400k WebVid +… See the full description on the dataset page: https://huggingface.co/datasets/ShareGPTVideo/train_video_and_instruction.stack-exchange-instruction
Dataset Card for "stack-exchange-instruction"
More Information needed
Instruction-Following-IFEval
SEA-IFEval
SEA-IFEval evaluates a model's ability to adhere to constraints provided in the prompt, for example beginning a response with a specific word/phrase or answering with a certain number of sections. It is based on IFEval and was manually translated by native speakers for Indonesian, Javanese, Sundanese, Thai, Tagalog, and Vietnamese.
Supported Tasks and Leaderboards
SEA-IFEval is designed for evaluating chat or instruction-tuned large language models (LLMs).… See the full description on the dataset page: https://huggingface.co/datasets/aisingapore/Instruction-Following-IFEval.trl-test-instructionck12-tqa-instruction
CK-12 TQA: Textbook Question Answering (Instruction Format)
Dataset Description
Dataset Summary
This is a reformatted version of the TQA (Textbook Question Answering) dataset, converted into an instruction-following format suitable for training and evaluating large language models on science question answering and multimodal reasoning tasks.
The TQA dataset consists of 1,076 lessons from Life Science, Earth Science, and Physical Science textbooks sourced from… See the full description on the dataset page: https://huggingface.co/datasets/notefill/ck12-tqa-instruction.opsd-instruction-scale-omni-full-v1-artifacts-publicinstruction-attack-outputsMalay-Dialect-Instructions
Malay dialect instruction including coding
Negeri Sembilan
QA
public transport QA,
Coding
CUDA coding,
Kedah
QA
infra QA,
Coding
Rust coding,
Kelantan
QA
Najib Razak QA,
Coding
Go coding,
Perak
QA
Anwar Ibrahim QA,
Coding
SQL coding,
Pahang
QA
Pendatang asing QA,
Coding
Typescript coding,
Terengganu… See the full description on the dataset page: https://huggingface.co/datasets/mesolitica/Malay-Dialect-Instructions.Turkish-Python-instruction
🚀 DİKKAT VERİ SETİ GÜNCELLENME SÜRECİNE ALINMIŞTIR LÜTFEN AÇIKLAMAYI OKUYUNUZ. Turkish Python & System Engineering Dataset (BYSISMO v2.0)
25 Kategorilik Büyük Türkçe Python & Sistem Mühendisliği Havuzu
📢 SÜRÜM & DOĞRULAMA DURUMU (VERSION ROADMAP)
v1.0 (Eski Arşiv - 289K / 8 Kategori): Yüksek kalite standartlarımız gereği yeniden yapılandırmaya alınmış ve dondurulmuştur.
v2.0 (Yeni Master Sürüm - 416K+ / 17 Kategori): Kodlar yalnızca sözdizimi… See the full description on the dataset page: https://huggingface.co/datasets/bysismo/Turkish-Python-instruction.instructions
Merged Instructions Dataset
Merged Dataset for the response of instructions.
instruction-speech-encodec-v1
Dataset Card for "Instruction Speech"
The largest open-source English speech instruction to text answer dataset
Dataset Overview
This dataset contains nearly 450,000 English speech instruction to text answer samples, using:
A subset of OpenHermes 2.5 with user's prompt length less than 64.
Audio generation using WhisperSpeech.
Tokenized using Encodec.
Usage
from datasets import load_dataset, Audio
# Load Instruction Speech dataset
dataset =… See the full description on the dataset page: https://huggingface.co/datasets/Menlo/instruction-speech-encodec-v1.X2I-mm-instruction
X2I Dataset
Project Page: https://vectorspacelab.github.io/OmniGen/
Github: https://github.com/VectorSpaceLab/OmniGen
Paper: https://arxiv.org/abs/2409.11340
Model: https://huggingface.co/Shitao/OmniGen-v1
To achieve robust multi-task processing capabilities, it is essential to train the OmniGen on large-scale and diverse datasets. However, in the field of unified image generation, a readily available dataset has yet to emerge. For this reason, we have curated a large-scale… See the full description on the dataset page: https://huggingface.co/datasets/yzwang/X2I-mm-instruction.natural-instructions-tokenized
Dataset Card for "natural-instructions-tokenized"
Here is the script used to tokenize the dataset:
import multiprocessing
from typing import Union
from datasets import DatasetDict, load_dataset
from transformers import LlamaTokenizer
# Find your available cores
num_cores = multiprocessing.cpu_count()
cutoff_len = 2048
tokenizer = LlamaTokenizer.from_pretrained("chainyo/alpaca-lora-7b")
tokenizer.padding_side = "left"
tokenizer.pad_token_id = (0)
prompt_template = {… See the full description on the dataset page: https://huggingface.co/datasets/chainyo/natural-instructions-tokenized.