datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
general-instruction-augmented-corpora
Instruction Pre-Training: Language Models are Supervised Multitask Learners (EMNLP 2024)
This repo contains the general instruction-augmented corpora (containing 200M instruction-response pairs covering 40+ task categories) used in our paper Instruction Pre-Training: Language Models are Supervised Multitask Learners.
We explore supervised multitask pre-training by proposing Instruction Pre-Training, a framework that scalably augments massive raw corpora with instruction-response… See the full description on the dataset page: https://huggingface.co/datasets/instruction-pretrain/general-instruction-augmented-corpora.genies_preferences
Dataset Card for "genie_dpo"
A conversion of the distribution from GENIES to open_pref_eval format.
Conversion code
GenesisII
QVAC Genesis II: Expanding the Largest Multi-domain Educational Synthetic Dataset for Pre-training
📖 Read the full blog post on Hugging Face | 🔗 Genesis I Dataset
QVAC Genesis II is a major expansion of the largest publicly available education-focused synthetic dataset for LLM pre-training and reasoning-centric post-training. Building upon Genesis I, it adds 10 new educational domains and introduces a novel Option-Level Reasoning Analysis data generation method, totaling 86… See the full description on the dataset page: https://huggingface.co/datasets/qvac/GenesisII.general-reasoning-ift-pairs
Reasoning-IFT Pairs (General Domain)
This dataset provides the largest set of IFT and Reasoning answers pairs for a set of general domain queries (cf: math-domain).It is based on the Infinity-Instruct dataset, an extensive and high-quality collection of instruction fine-tuning data.
We curated 900k queries from the 7M_core subset of Infinity-Instruct, which covers multiple domains including general knowledge, commonsense Q&A, coding, and math.For each query… See the full description on the dataset page: https://huggingface.co/datasets/Scale-or-Reason/general-reasoning-ift-pairs.FIGNEWS_generated_queriesThis repository contains the FIGNEWS dataset with predicted queries, a core component used in the paper QAEncoder: Towards Aligned Representation Learning in Question Answering Systems.
The official implementation and related code are available on GitHub: https://github.com/IAAR-Shanghai/QAEncoder
Introduction
Modern QA systems entail retrieval-augmented generation (RAG) for accurate and trustworthy responses. However, the inherent gap between user queries and relevant documents… See the full description on the dataset page: https://huggingface.co/datasets/zr-wang/FIGNEWS_generated_queries.general-reasoning-ift-pairs
Reasoning-IFT Pairs (General Domain)
This dataset provides the largest set of IFT and Reasoning answers pairs for a set of general domain queries (cf: math-domain).It is based on the Infinity-Instruct dataset, an extensive and high-quality collection of instruction fine-tuning data.
We curated 900k queries from the 7M_core subset of Infinity-Instruct, which covers multiple domains including general knowledge, commonsense Q&A, coding, and math.For each query, we… See the full description on the dataset page: https://huggingface.co/datasets/Sidsidney/general-reasoning-ift-pairs.General-Knowledge
Dataset Card for Dataset Name
Dataset Summary
The dataset is a collection of questions and answers themed on general facts and reasoning. The dataset is divided into two features - 'Question' and 'Answer'.
It is meant to be used for training a model to be good at general knowledge and reasoning. This dataset is inspired from the Alpaca dataset, and infact contains a subset of the alpaca dataset in itself.
Distribution
The distribution of the… See the full description on the dataset page: https://huggingface.co/datasets/MuskumPillerum/General-Knowledge.GeneratingQuestions
HVU_QA
HVU_QA is an open-source Vietnamese Question-Context-Answer (QCA) corpus, accompanied by supporting tools, created to facilitate the development of FAQ-style question generation and question answering systems, particularly for low-resource language settings. The dataset was developed by a research team at Hung Vuong University, Phu Tho, Vietnam, led by Dr. Ha Nguyen, Deputy Head of the Department of Engineering Technology. HVU_QA was constructed using a fully automated… See the full description on the dataset page: https://huggingface.co/datasets/DANGDOCAO/GeneratingQuestions.nortruthfulqa_gen
Dataset Card for NorTruthfulQA Generation
Dataset Details
Dataset Description
NorTruthfulQA Generation is an open-ended question answering (QA) dataset designed to evaluate whether a language model generate answers that convey false beliefs or misconceptions. The dataset contains 471 examples in both written standards of Norwegian: Bokmål and Nynorsk (the minority variant). Each example includes a question, 1 to 12 correct and incorrect reference answers, and a… See the full description on the dataset page: https://huggingface.co/datasets/ltg/nortruthfulqa_gen.chinese-biomedicine-and-genomics-open-intelligence
🔬 Chinese Biomedicine, Cell Therapy & Genomics Open Intelligence Dataset
Curated open intelligence dataset providing English briefs, clinical trial benchmarks, verified abstracts, and DOIs of frontier Chinese research in Cellular Therapeutics, Gene Editing, ADCs, and NMPA Clinical Approvals.
[!IMPORTANT]
Data Completeness & Research Authenticity Notice:
Included in this Hugging Face Open Dataset: English structured abstracts, core quantitative takeaways, author… See the full description on the dataset page: https://huggingface.co/datasets/simpleG2023/chinese-biomedicine-and-genomics-open-intelligence.scientific-verification
Scientific Verification Benchmark: NMC Cathodes
Dataset summary
The benchmark contains 50 scientific claims about NMC (lithium nickel manganese cobalt oxide) battery cathodes. Each claim is answered by Claude Opus 5, GPT 5.6 Luna and Gemini 3.1 Pro using a set of 20 open-access papers, producing 150 scored answers. The accompanying reference set contains 1,991 experiment-grounded measurements curated from 227 open-access papers, with experimental conditions and… See the full description on the dataset page: https://huggingface.co/datasets/GenData-Research/scientific-verification.generalization-dynamics-evals
Generalization Dynamics — Main Eval Suite
Prepared test sets for the 6 main evaluation families from
Generalization dynamics across fine-tuning
(Table 1).
Use with the unified runner:
https://github.com/jiaxin-wen/FT-generalization/tree/main/release
from huggingface_hub import snapshot_download
root = snapshot_download(
repo_id="jiaxin-wen/generalization-dynamics-evals", repo_type="dataset")
Or browse a single task (the dataset viewer shows all configs):
from datasets… See the full description on the dataset page: https://huggingface.co/datasets/jiaxin-wen/generalization-dynamics-evals.Japanese-RAG-Generator-Benchmark
Japanese RAG Generator Benchmark: 日本語 RAG における Generator 評価ベンチマーク
Japanese RAG Generator Benchmark (J-RAGBench) は日本語RAGにおけるGeneratorに用いるLLMの評価データセットを提供する。
実運用時のRAGに求められる多様な評価カテゴリを同一条件下で評価可能であり、複数の評価カテゴリが同時に出現する問題が含まれるQAデータセットを人手および、補助的にOpenAI API(gpt-4.1-2025-04-14)を用いて構築した。
J-RAGBenchの評価カテゴリ
Integration: 2~3文書程度の複数の情報源から適切な根拠を抽出・統合して回答を導く
Reasoning: 抽出された情報を踏まえて多段階の推論や数値計算などを実行する
Logical: 質問・関連文書間での語彙や表現の差異を解釈し、適切な回答を導く
Table:… See the full description on the dataset page: https://huggingface.co/datasets/neoai-inc/Japanese-RAG-Generator-Benchmark.InterviewForge_GenDS
Synthetic Data Generation
Model & Infrastructure
The dataset was generated using the mistral:latest Large Language Model running locally via the Ollama framework. This model was explicitly selected because it balances advanced reasoning capabilities with hardware efficiency, allowing the execution of 10,944 complex generation requests entirely locally on an RTX 3080 GPU without incurring API costs. Additionally, Mistral demonstrated exceptional reliability in… See the full description on the dataset page: https://huggingface.co/datasets/Davichick/InterviewForge_GenDS.OCR-MetaReasoning
OCR-MetaReasoning Benchmark: Evaluating the Meta-Reasoning Ability of MLLMs in Text-Rich Image Understanding
Gengxu Li1, Yuan Wu1*, Yi Chang1,2,3
1 School of Artificial Intelligence, Jilin University 2 Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, MOE, China
3 International Center of Future Science, Jilin University
Overview ·
Task ·
Dataset ·
Evaluation ·
Citation
OCR-MetaReasoning is a controlled benchmark for testing… See the full description on the dataset page: https://huggingface.co/datasets/GengxuLi123/OCR-MetaReasoning.chatgpt4-malaysian-general-qa
Synthetic Malaysian QA
Generated common QA using ChatGPT4 based on Malaysia topics, notebooks at https://github.com/mesolitica/malaysian-dataset/tree/master/question-answer/chatgpt4-synthetic-malaysian-qa
General Malaysia topics
malaysian-general-qa.jsonl, 20396 rows, 28.6 MB.
malaysian-general-qa-v2.jsonl, 5294 rows, 8.05 MB.
malaysian-general-qa-v3.jsonl, 1368 rows, 5.09 MB.
malaysian-general-qa-v4.jsonl, 7733 rows. 36.2 MB.
malaysian-general-qa-v5.jsonl, 6363 rows… See the full description on the dataset page: https://huggingface.co/datasets/mesolitica/chatgpt4-malaysian-general-qa.Aloe-Beta-General-Collection
Aloe-Beta-Medical-Collection
Collection of curated general datasets used to fine-tune Aloe-Beta.
Dataset Details
Dataset Description
We curated data from many publicly available general instruction tuning data sources (QA format). It consists of 400k instructions including:
Coding, math, data analysis, STEM, etc.
Function calling
Creative writing, advice seeking… See the full description on the dataset page: https://huggingface.co/datasets/HPAI-BSC/Aloe-Beta-General-Collection.General365_Public
🧩 General365: Benchmarking General Reasoning in LLMs Across Diverse and Challenging Tasks
📃 Paper • 🌐 Project Page • 🏆 Leaderboard •
💻 Github
📖 Introduction
We present General365, a highly challenging and diverse benchmark for evaluating the general reasoning capabilities in LLMs.
"General Reasoning" refers to reasoning tasks that depend exclusively on general knowledge.
We define general knowledge as knowledge within the K-12 scope (such as common sense… See the full description on the dataset page: https://huggingface.co/datasets/meituan-longcat/General365_Public.TRQA
Dataset Card for TRQA
TRQA is an evaluation dataset designed to systematically assess the comprehensive capacities of OriGene and other multi-agent
frameworks in the field of therapeutic target discovery. TRQA focuses on key areas such as fundamental biology, disease
biology, pharmacology, and clinical medicine, aiming to evaluate the ability to conduct effective planning, gather useful information,
choose appropriate tools, reason to scientific conclusions, and critically… See the full description on the dataset page: https://huggingface.co/datasets/GENTEL-Lab/TRQA.NeuroBio-GenZ-1K
NeuroBio GenZ 1K
Around 1000 neuroscience and biology questions, answered like your smartest friend is texting you back, not like a textbook is talking at you.
"Why does doomscrolling give me dopamine?" gets answered in three sentences, casual tone, real neuroscience terms (nucleus accumbens, not "reward center"), zero fluff.
Why did you make this?
Because there's genuinely not that much high quality neuroscience and biology data on Hugging Face that isn't either… See the full description on the dataset page: https://huggingface.co/datasets/luka0x12/NeuroBio-GenZ-1K.Generated-Recovery-Support-Dialogues
# Empathetic Conversations for Addiction Recovery Support Dataset
Dataset Description
This dataset contains synthetically generated conversational examples between a user discussing their addiction recovery journey and an AI assistant designed to be empathetic, supportive, non-judgmental, and encouraging. The conversations are in English and cover various stages and aspects of the recovery process, following established therapeutic guidelines and models.
The dataset is… See the full description on the dataset page: https://huggingface.co/datasets/filippo19741974/Generated-Recovery-Support-Dialogues.telco-gaia
Telco-GAIA
A GAIA-style benchmark for AI agents operating over a real telecom operator's
website snapshot plus a synthetic customer database. 100 tasks across 7
categories: Pricing, Miscellaneous, Images, Web Archives, PDF, PDF Visual,
Database.
Agents read questions.json + environment.md, browse the local website
(:8080) and query the database API (:8081), and produce a GAIA-compatible
submission.json.
What's here
File
What… See the full description on the dataset page: https://huggingface.co/datasets/kaust-generative-ai/telco-gaia.SWE-QA-Benchmark
SWE-QA Benchmark
A comprehensive benchmark dataset for Software Engineering Question Answering, containing 720 questions across 15 popular Python repositories.
Dataset Summary
Total Questions: 720
Repositories: 15
Format: JSONL (JSON Lines)
Fields: question, answer
Repository Coverage
Each repository contains 48 questions:
astropy
conan
django
flask
matplotlib
pylint
pytest
reflex
requests
scikit-learn
sphinx
sqlfluff
streamlink
sympy
xarray… See the full description on the dataset page: https://huggingface.co/datasets/GeniusWondering/SWE-QA-Benchmark.dataset-ohada-droit-commercial-general-echantillon
Dataset OHADA — Droit Commercial Général (AUDCG) — Échantillon
Description
Échantillon de 10 entrées extraites d'un dataset de fine-tuning juridique en cours de conception, portant sur l'Acte Uniforme relatif au Droit Commercial Général (AUDCG) — le texte fondamental du statut du commerçant, des actes de commerce, de la preuve et de la prescription en matière commerciale dans l'espace OHADA (Organisation pour l'Harmonisation en Afrique du Droit des Affaires — 17… See the full description on the dataset page: https://huggingface.co/datasets/Bisilivan/dataset-ohada-droit-commercial-general-echantillon.genolator-v1-qa
Genolator V1 — Multimodal Gene Function QA
Question–answer pairs about human gene function, paired with precomputed embeddings of
three modalities per gene: the coding DNA sequence, the amino acid sequence, and the
predicted 3D protein structure. It is the dataset used to train and evaluate Genolator
V1, a model that projects those embeddings into the token embedding space of a
biomedical Llama-3 and answers questions about a gene without ever seeing its name or
its raw… See the full description on the dataset page: https://huggingface.co/datasets/CHGGM-Aachen/genolator-v1-qa.Genshin-Impact-SFTgujarati-general-purpose-instruction
Gujarati General-Purpose Instruction Dataset (GGJI v1)
Dataset Summary
GGJI v1 (Gujarati General-Purpose Instruction v1) is a large-scale, high-quality supervised fine-tuning (SFT) dataset designed to train instruction-following language models in Gujarati. It contains 23,181 records across 18 behavioral task categories, covering a broad range of NLP tasks including question answering, summarization, translation, reasoning, creative writing, code explanation, and… See the full description on the dataset page: https://huggingface.co/datasets/tkdonda/gujarati-general-purpose-instruction.Skill2-Bench
Skill²-Bench
Skill²-Bench is a benchmark of multi-step tasks that force LLMs to switch between skills, introduced in the paper "Toward Skill-Native LLMs: Skill Entropy for Benchmarking and Training Long-Horizon Reasoning".
Long-horizon tasks require models to switch between skills, not just execute a single skill well. Each Skill²-Bench task embeds a sequence of 2–10 steps in a coherent real-world scenario, where consecutive steps draw on different skills (e.g., algorithm design… See the full description on the dataset page: https://huggingface.co/datasets/Gen-Verse/Skill2-Bench.vi_question_generationQuestion-Answering-Generation-Choices
The dataset is a merged compilation of QuAIL, RACE, and Cosmos QA datasets,
having undergone preprocessing.