datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
textbooks
The Textbooks Corpus in MedRAG
This HF dataset contains the chunked snippets from the Textbooks corpus used in MedRAG. It can be used for medical Retrieval-Augmented Generation (RAG).
Dataset Details
Dataset Descriptions
Textbooks is a collection of 18 widely used medical textbooks, which are important references for students taking the United States Medical Licensing Examination (USLME).
In MedRAG, the textbooks are processed as chunks with no more than 1000… See the full description on the dataset page: https://huggingface.co/datasets/MedRAG/textbooks.Reverse-Text-RL
Reverse-Text-RL
A small, scrappy RL dataset used in prime-rl's CI to debug RL training asking a model to reverse small sentences character-by-character. Follows the general format of PrimeIntellect/Reverse-Text-SFT
The following script was used to generate the dataset.
from datasets import Dataset, load_dataset
dataset = load_dataset("willcb/R1-reverse-wikipedia-paragraphs-v1-1000", split="train")
prompt = "Reverse the text character-by-character. Put your answer in… See the full description on the dataset page: https://huggingface.co/datasets/PrimeIntellect/Reverse-Text-RL.synthetic_text_to_sql
Image generated by DALL-E. See prompt for more details
synthetic_text_to_sql
gretelai/synthetic_text_to_sql is a rich dataset of high quality synthetic Text-to-SQL samples,
designed and generated using Gretel Navigator, and released under Apache 2.0.
Please see our release blogpost for more details.
The dataset includes:
105,851 records partitioned into 100,000 train and 5,851 test records
~23M total tokens, including ~12M SQL tokens
Coverage across 100 distinct… See the full description on the dataset page: https://huggingface.co/datasets/gretelai/synthetic_text_to_sql.Reverse-Text-SFT
Reverse-Text-SFT
A small, scrappy SFT dataset used for warming up a small model (e.g. Qwen/Qwen3-0.6B) for RL training. Contains examples in prompt-completion chat format of reversing 5-20 words of text character-by-character. The raw sentences were processed from willcb/R1-reverse-wikipedia-paragraphs-v1-1000.
The following script was used to generate the dataset.
from datasets import Dataset, load_dataset
dataset = load_dataset("willcb/R1-reverse-wikipedia-paragraphs-v1-1000"… See the full description on the dataset page: https://huggingface.co/datasets/PrimeIntellect/Reverse-Text-SFT.nntp-text-387m
NNTP Discussion Archives
A large-scale collection of text discussions from public NNTP (Network News Transfer Protocol) newsgroups spanning over two decades.
Dataset Statistics
Metric
Value
Total messages
386,629,949
Unique newsgroups
159,345
Date range
2002 - 2026
Total size
~191 GB (compressed)
File format
Parquet (ZSTD)
Number of files
256
Average content length
~1,400 characters
Schema
Column
Type
Description… See the full description on the dataset page: https://huggingface.co/datasets/nyuuzyou/nntp-text-387m.python-text-copilot-training-instruct-ai-research-2024-02-03
Python Copilot Instructions on How to Code using Alpaca and Yaml
Training and test datasets for building coding multimodal models that understand how to use the open source GitHub projects for the Agora Open Source AI Research Lab:
Agora GitHub Organization
Agora Hugging Face
This dataset is the 2024-02-03 update for the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details… See the full description on the dataset page: https://huggingface.co/datasets/matlok/python-text-copilot-training-instruct-ai-research-2024-02-03.text-sft-questions-answers-only
text-sft: Questions and Answers
This dataset consists of question-and-answer pairs generated from short excerpts drawn from Wikipedia, Cosmopedia, and FineWeb-Edu. It is an adapted version of agentlans/text-sft.
Overview
The dataset provides compact examples of English question-and-answer relationships that can help models learn linguistic patterns, syntactic structures, and semantic associations between questions and their corresponding answers.
Intended Use… See the full description on the dataset page: https://huggingface.co/datasets/agentlans/text-sft-questions-answers-only.TextVQA_GT_bbox
TextVQA validation set with grounding truth bounding box
The dataset used in the paper MLLMs Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLMs for studying MLLMs' attention patterns.
The dataset is sourced from TextVQA and annotated manually with ground-truth bounding boxes.
We consider questions with a single area of interest in the image so that 4370 out of 5000 samples are kept.
Citation
If you find our paper and code useful… See the full description on the dataset page: https://huggingface.co/datasets/jrzhang/TextVQA_GT_bbox.python-text-copilot-training-instruct
Python Copilot Instructions on How to Code using Alpaca and Yaml
This dataset is a subset of the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details
Each row contains python code, either a class method or a global function, imported modules, base classes (if any), exceptions (ordered based off the code), returns (ordered based off the code), arguments (ordered based off the… See the full description on the dataset page: https://huggingface.co/datasets/matlok/python-text-copilot-training-instruct.gretel-synthetic-text-to-sql
Fork of gretelai/synthetic_text_to_sql
The gretelai/synthetic_text_to_sql dataset is a large, Apache 2.0 licensed, synthetic Text-to-SQL dataset consisting of 105,851 high-quality records across 100 diverse domains, designed for training language models. It includes comprehensive SQL tasks with varying complexities, database contexts, natural language explanations, and contextual tags, outperforming existing datasets in SQL correctness and standards compliance.
python-text-training-instruct-ai
Python Copilot Instructions on How to Code using Alpaca and Yaml
Training and test datasets for building coding multimodal models that understand how to use the open source GitHub projects for the Agora Open Source AI Research Lab:
Agora GitHub Organization
Agora Hugging Face
This dataset is the 2024-02-03 update for the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details… See the full description on the dataset page: https://huggingface.co/datasets/DevShubham/python-text-training-instruct-ai.judaic-texts-corpus
Judaic Texts Corpus
Dataset Summary
Judaic Texts Corpus is a machine-readable Hebrew and Aramaic corpus of Judaic
texts derived from the Otzaria library release archives. It is intended for
language-model training, retrieval, search, digital humanities research, and
other NLP workflows that need structured access to rabbinic and traditional
Jewish texts.
The current dataset build is produced from the official
Otzaria/otzaria-library release
assets, which package… See the full description on the dataset page: https://huggingface.co/datasets/NHLOCAL/judaic-texts-corpus.TextFacts
Look at the facts, and decide whether or not you'd like to use this.
ARCHIVE-TEXT-URLS
Internet Archive English Text URLs Dataset
Dataset Description
This dataset contains 11,151,637 direct download URLs to OCR-processed text files from the Internet Archive's digital library. All entries are English-language texts spanning books, documents, historical records, and various other written materials.
Dataset Summary
Total Rows: 11,151,637
Language: English
Source: Internet Archive
Format: CSV with metadata and direct text file URLs
Text… See the full description on the dataset page: https://huggingface.co/datasets/Navanjana/ARCHIVE-TEXT-URLS.python-text-copilot-training-instruct-ai-research
Building an AI Copilot Dataset to help keep up with Leading AI Research
This is a specialized, instruction dataset for training python coding assistants on how to code from leading AI/ML open source repositories (2.3M coding samples).
This dataset is a subset of the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details
This dataset holds the latest coding changes from >1159… See the full description on the dataset page: https://huggingface.co/datasets/matlok/python-text-copilot-training-instruct-ai-research.medical_textbooks_mcq
Medical Textbooks MCQs Dataset
This dataset is derived from the Textbooks corpus chunks found in the MedRAG/textbooks dataset. It augments the original text snippets with synthetically generated Multiple Choice Questions (MCQs) in JSON format, suitable for fine-tuning or evaluating language models on medical MCQ generation tasks.
Dataset Details
Dataset Description
The source data consists of text snippets from the Textbooks corpus, a collection of 18 widely… See the full description on the dataset page: https://huggingface.co/datasets/asanchez75/medical_textbooks_mcq.python-text-copilot-training-instruct-ai-research-2024-02-10
Python Copilot Instructions on How to Code using Alpaca and Yaml
Training and test datasets for building coding multimodal models that understand how to use the open source GitHub projects for the multimodal Qwen AI project:
Qwen
Qwen Agent
Qwen VL Chat
Qwen Audio
This dataset is the 2024-02-10 update for the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details
Each row… See the full description on the dataset page: https://huggingface.co/datasets/matlok/python-text-copilot-training-instruct-ai-research-2024-02-10.python-text-copilot-training-instruct-ai-research-2024-02-11
Python Copilot Instructions on How to Code using Alpaca and Yaml
Training and test datasets for building coding multimodal models that understand how to use the open source GitHub projects for the Autogen and multimodal Qwen AI project:
Qwen
Qwen Agent
Qwen VL Chat
Qwen Audio
This dataset is the 2024-02-11 update for the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details… See the full description on the dataset page: https://huggingface.co/datasets/matlok/python-text-copilot-training-instruct-ai-research-2024-02-11.medrag_textbooks
The Textbooks Corpus in MedRAG
This HF dataset contains the chunked snippets from the Textbooks corpus used in MedRAG. It can be used for medical Retrieval-Augmented Generation (RAG).
Dataset Details
Dataset Descriptions
Textbooks is a collection of 18 widely used medical textbooks, which are important references for students taking the United States Medical Licensing Examination (USLME).
In MedRAG, the textbooks are processed as chunks with no more than 1000… See the full description on the dataset page: https://huggingface.co/datasets/minsu/medrag_textbooks.python-text-copilot-training-instruct-ai-research-2024-01-27
Python Copilot Instructions on How to Code using Alpaca and Yaml
This dataset is the 2024-01-27 update for the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details
Each row contains python code, either a class method or a global function, imported modules, base classes (if any), exceptions (ordered based off the code), returns (ordered based off the code), arguments (ordered… See the full description on the dataset page: https://huggingface.co/datasets/matlok/python-text-copilot-training-instruct-ai-research-2024-01-27.textbook
Dataset Card for Dataset Name
This dataset card aims to be a base template for new datasets. It has been generated using this raw template.
Dataset Details
Dataset Description
Curated by: [More Information Needed]
Funded by [optional]: [More Information Needed]
Shared by [optional]: [More Information Needed]
Language(s) (NLP): [More Information Needed]
License: [More Information Needed]
Dataset Sources [optional]
Repository: [More… See the full description on the dataset page: https://huggingface.co/datasets/kiran475/textbook.greatnorth-canada-federal-laws-text
Great North Canada Federal Laws Text Corpus (Expanded)
235,794 training examples from the complete current body of Canadian consolidated federal Acts and Regulations.
This is a significantly expanded version of the corpus, now including:
All consolidated Acts
All consolidated Regulations
Both English and French versions where available
Better chunking optimized for LLM training
Data Characteristics
Section/provision-level text from official Government of Canada… See the full description on the dataset page: https://huggingface.co/datasets/GreatNorthCollective/greatnorth-canada-federal-laws-text.html-query-text-HtmlRAG
html-query-text-HtmlRAG
Warning: This dataset is under development and its content is subject to change!
This dataset is a processed and cleaned version of the zstanjj/HtmlRAG-train dataset. It has been specifically prepared for task of HTML cleaning.
🚀 Supported Tasks
This dataset is primarily designed for:
HTML Cleaning: Training models to take the messy html as input and generate the cleaned_html or cleaned_text as output.
Question Answering: Training models to… See the full description on the dataset page: https://huggingface.co/datasets/williambrach/html-query-text-HtmlRAG.gsm8k-train-nomic-text-v1.5
Overview
Dataset containing embeddings / classification information for GSM8K
project02_textming_dataset
Dataset for Project 02 (Vietnamese Question Answering) - Text Mining and Application - FIT@HCMUS - 2024
Original dataset: Kaggle-CSC15105
How to load dataset?
!pip install transformers datasets
from datasets import load_dataset
hf_dataset = "nguyennghia0902/project02_textming_dataset"
load_raw_data = = load_dataset(hf_dataset, d
data_files={
'train': 'raw_data/train.json'… See the full description on the dataset page: https://huggingface.co/datasets/nguyennghia0902/project02_textming_dataset.textbooks
The Textbooks Corpus in MedRAG
This HF dataset contains the chunked snippets from the Textbooks corpus used in MedRAG. It can be used for medical Retrieval-Augmented Generation (RAG).
Dataset Details
Dataset Descriptions
Textbooks is a collection of 18 widely used medical textbooks, which are important references for students taking the United States Medical Licensing Examination (USLME).
In MedRAG, the textbooks are processed as chunks with no more than 1000… See the full description on the dataset page: https://huggingface.co/datasets/AdarshDS/textbooks.EarthScience-Text-LLM-20K-90-10
EarthScience-Text-LLM-20K-90-10
This is a pure-text Earth-science corpus unified from three non-overlapping upstream datasets:
Ekimetrics/climateqa-ipcc-ipbes-reports-1.0: climate and IPCC/IPBES report chunks.
GeoGPT-Research-Project/GeoGPT-CoT-QA: geoscience question-answer reasoning.
gremlin97/RemoteSensingCorpus: remote-sensing and geospatial machine-learning text.
Files and Split
The previous preprocessing outputs were merged into a 23,098-record pool and… See the full description on the dataset page: https://huggingface.co/datasets/moTcream/EarthScience-Text-LLM-20K-90-10.bird-text2sql-bench
Dataset Card for bird-text2sql-bench
bird-text2sql-bench 是 BIRD（BIg Bench for Large-Scale Database Grounded Text-to-SQL） 官方訓練集之 OpenAI Messages 格式版本，共 9,428 筆。相較於 Spider 1.0，BIRD 使用真實大型資料庫（70 個，涵蓋電商、運動、教育、醫療等 37+ 領域），並提供 evidence（數值提示）欄位，本資料集將 evidence 以 ### Hint 段落併入 user prompt，形成可直接餵入 SFT pipeline 之 system / user / assistant 三 role 對話。除原生之 messages 欄位外，另拆解出獨立之 system / user / assistant 字串欄位，可同時作為 SFT 語料與 benchmark evaluation pipeline 之直接輸入。
Dataset Details… See the full description on the dataset page: https://huggingface.co/datasets/lianghsun/bird-text2sql-bench.jfk-files-text
National Archives JFK Files Text Dataset
This dataset contains extracted text from the JFK assassination records released by the National Archives. The dataset preserves the original directory structure from archive.gov while providing significant performance and storage benefits for data analysis, AI applications, and large-scale processing.
Dataset Structure
The dataset is structured with the following columns:
Column
Description
year
The release year of the… See the full description on the dataset page: https://huggingface.co/datasets/mysocratesnote/jfk-files-text.TextCodeDepot
Dataset description:
The Python Code Chatbot dataset is a collection of Python code snippets extracted from various publicly available datasets and platforms. It is designed to facilitate training conversational AI models that can understand and generate Python code. The dataset consists of a total of 1,37,183 prompts, each representing a dialogue between a human and an AI Scientist.
Prompt Card:
Each prompt in the dataset follows a specific format known as the "Prompt… See the full description on the dataset page: https://huggingface.co/datasets/anujsahani01/TextCodeDepot.