datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
Reverse-Text-RL
Reverse-Text-RL
A small, scrappy RL dataset used in prime-rl's CI to debug RL training asking a model to reverse small sentences character-by-character. Follows the general format of PrimeIntellect/Reverse-Text-SFT
The following script was used to generate the dataset.
from datasets import Dataset, load_dataset
dataset = load_dataset("willcb/R1-reverse-wikipedia-paragraphs-v1-1000", split="train")
prompt = "Reverse the text character-by-character. Put your answer in… See the full description on the dataset page: https://huggingface.co/datasets/PrimeIntellect/Reverse-Text-RL.synthetic_text_to_sql
Image generated by DALL-E. See prompt for more details
synthetic_text_to_sql
gretelai/synthetic_text_to_sql is a rich dataset of high quality synthetic Text-to-SQL samples,
designed and generated using Gretel Navigator, and released under Apache 2.0.
Please see our release blogpost for more details.
The dataset includes:
105,851 records partitioned into 100,000 train and 5,851 test records
~23M total tokens, including ~12M SQL tokens
Coverage across 100 distinct… See the full description on the dataset page: https://huggingface.co/datasets/gretelai/synthetic_text_to_sql.TextbookReasoning
MegaScience: Pushing the Frontiers of Post-Training Datasets for Science Reasoning
Dataset Description
Scientific reasoning is critical for developing AI scientists and supporting human researchers in advancing the frontiers of natural science discovery. However, the open-source community has primarily focused on mathematics and coding while neglecting the scientific domain, largely due to the absence of open, large-scale, high-quality, verifiable scientific reasoning… See the full description on the dataset page: https://huggingface.co/datasets/MegaScience/TextbookReasoning.text-dataset-tiny-code-script-py-format
USED of tahamajs/medicine_ds_persian for .parquet file
USED of Alijafarixcs2/persian-it-llama2-2k for .parquet file
USED of Abirate/english_quotes for .jsonl file
NEW FILES (05/12/2025)
NEW FILES (12/26/2025)
NEW FILES (02/15/2026)
Reverse-Text-SFT
Reverse-Text-SFT
A small, scrappy SFT dataset used for warming up a small model (e.g. Qwen/Qwen3-0.6B) for RL training. Contains examples in prompt-completion chat format of reversing 5-20 words of text character-by-character. The raw sentences were processed from willcb/R1-reverse-wikipedia-paragraphs-v1-1000.
The following script was used to generate the dataset.
from datasets import Dataset, load_dataset
dataset = load_dataset("willcb/R1-reverse-wikipedia-paragraphs-v1-1000"… See the full description on the dataset page: https://huggingface.co/datasets/PrimeIntellect/Reverse-Text-SFT.theekkathir-text-datasettheekkathir-text-dataset <-> தீக்கதிர் தரவுத்தொகுப்பு
Click above button to view GitHub Repository
இலக்கு:
இந்த திட்டத்தின் இலக்கு தீக்கதிர் இதழின் செய்தி கட்டுரைகளை தரவுத்தொகுப்பாக மாற்றுவதாகும், இது இயற்கை மொழி பதிவு (NLP) மற்றும் LLM ஆராய்ச்சி நோக்கங்களுக்கு பயன்படுத்தப்படலாம்.
Goal:
The goal of the project is to convert news articles from theekkathir magazine into dataset, which can be used for Natural Language Processing (NLP) and LLM research purposes… See the full description on the dataset page: https://huggingface.co/datasets/aiwithvarun7/theekkathir-text-dataset.nntp-text-387m
NNTP Discussion Archives
A large-scale collection of text discussions from public NNTP (Network News Transfer Protocol) newsgroups spanning over two decades.
Dataset Statistics
Metric
Value
Total messages
386,629,949
Unique newsgroups
159,345
Date range
2002 - 2026
Total size
~191 GB (compressed)
File format
Parquet (ZSTD)
Number of files
256
Average content length
~1,400 characters
Schema
Column
Type
Description… See the full description on the dataset page: https://huggingface.co/datasets/nyuuzyou/nntp-text-387m.TextInsightBench
TextInsightBench
English | 简体中文
A natural-language data-mining benchmark for agents: 50 tasks, 435,000 task documents and 944,468 unlabeled learning documents.
Each task provides 5,000 or 10,000 texts and a research objective. Agents choose the patterns, populations and comparisons to investigate, then submit up to three findings with complete document assignments, exact quotations, statistics, counterexamples and limitations. Any analysis method is allowed.
Contents… See the full description on the dataset page: https://huggingface.co/datasets/CodeSoulco/TextInsightBench.Nepali-Text-Corpus
Nepali Text Corpus
Overview
Nepali-Text-Corpus is a comprehensive collection of approximately 6.4 million articles in the
Nepali language. This dataset is the largest text dataset on Nepali Language. It encompasses a
diverse range of text types, including news articles, blogs, and more, making it an invaluable
resource for researchers, developers, and enthusiasts in the fields of Natural Language Processing (NLP)
and computational linguistics.
Dataset… See the full description on the dataset page: https://huggingface.co/datasets/IRIIS-RESEARCH/Nepali-Text-Corpus.FineWeb2-HQ-zh-text
Original Database
https://huggingface.co/datasets/epfml/FineWeb2-HQ
Work
keep cmn_Hani with text field
github-jupyter-code-to-text
Dataset description
This dataset consists of sequences of Python code followed by a a docstring explaining its function. It was constructed by concatenating code and text pairs
from this dataset that were originally code and markdown cells in Jupyter Notebooks.
The content of each example the following:
[CODE]
"""
Explanation: [TEXT]
End of explanation
"""
[CODE]
"""
Explanation: [TEXT]
End of explanation
"""
...
How to use it
from datasets import load_dataset
ds =… See the full description on the dataset page: https://huggingface.co/datasets/codeparrot/github-jupyter-code-to-text.python-text-copilot-training-instruct-ai-research-2024-02-03
Python Copilot Instructions on How to Code using Alpaca and Yaml
Training and test datasets for building coding multimodal models that understand how to use the open source GitHub projects for the Agora Open Source AI Research Lab:
Agora GitHub Organization
Agora Hugging Face
This dataset is the 2024-02-03 update for the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details… See the full description on the dataset page: https://huggingface.co/datasets/matlok/python-text-copilot-training-instruct-ai-research-2024-02-03.SQaLe-text-to-SQL-dataset
🧮 SQALE: A Large-Scale Semi-Synthetic Dataset
SQALE is a large-scale, semi-synthetic Text-to-SQL dataset grounded in real-world database schemas.
It was designed to push the boundaries of natural language to SQL generation, combining realistic schema diversity, complex query structures, and linguistically varied natural language questions.
The dataset was introduced in the paper SQaLe: A Large Text-to-SQL Corpus Grounded in Real Schemas. The code for the generation pipeline of this… See the full description on the dataset page: https://huggingface.co/datasets/trl-lab/SQaLe-text-to-SQL-dataset.TextVQA_GT_bbox
TextVQA validation set with grounding truth bounding box
The dataset used in the paper MLLMs Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLMs for studying MLLMs' attention patterns.
The dataset is sourced from TextVQA and annotated manually with ground-truth bounding boxes.
We consider questions with a single area of interest in the image so that 4370 out of 5000 samples are kept.
Citation
If you find our paper and code useful… See the full description on the dataset page: https://huggingface.co/datasets/jrzhang/TextVQA_GT_bbox.tiny-textbooks
Textbook-like Dataset: A High-Quality Resource for Small Language Models
The idea is simply inspired by the Textbooks Are All You Need II: phi-1.5 technical report paper. The source texts in this dataset have been gathered and carefully select the best of the falcon-refinedweb and minipile datasets to ensure the diversity, quality while tiny in size. The dataset was synthesized using 4x3090 Ti cards over a period of 500 hours, thanks to Nous-Hermes-Llama2-13b finetuned model.
Why… See the full description on the dataset page: https://huggingface.co/datasets/nampdn-ai/tiny-textbooks.python-text-copilot-training-instruct
Python Copilot Instructions on How to Code using Alpaca and Yaml
This dataset is a subset of the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details
Each row contains python code, either a class method or a global function, imported modules, base classes (if any), exceptions (ordered based off the code), returns (ordered based off the code), arguments (ordered based off the… See the full description on the dataset page: https://huggingface.co/datasets/matlok/python-text-copilot-training-instruct.ultradata-math-textbook-exercise-ar
ultradata-math-textbook-exercise-ar
Arabic translation of the English portion of UltraData-Math, config UltraData-Math-L3-Textbook-Exercise-Synthetic: synthetic textbook-style content and exercises generated around specific mathematical knowledge points. Translated with the midtrans pipeline: text is segmented into prose and verbatim blocks (LaTeX, code, tables, and inline non-translatables are masked and never sent to the model, so formulas cannot be mangled), prose is… See the full description on the dataset page: https://huggingface.co/datasets/SultanR/ultradata-math-textbook-exercise-ar.UltraData-Math-L3-Textbook-Exercise-Synthetic-split
UltraData-Math L3 Textbook Exercise Synthetic Split
Source dataset: openbmb/UltraData-Math
Source config: UltraData-Math-L3-Textbook-Exercise-Synthetic
Each row contains:
uid
question
answer
The original content field was split using the literal markers
The exercise: and The solution:.
gretel-synthetic-text-to-sql
Fork of gretelai/synthetic_text_to_sql
The gretelai/synthetic_text_to_sql dataset is a large, Apache 2.0 licensed, synthetic Text-to-SQL dataset consisting of 105,851 high-quality records across 100 diverse domains, designed for training language models. It includes comprehensive SQL tasks with varying complexities, database contexts, natural language explanations, and contextual tags, outperforming existing datasets in SQL correctness and standards compliance.
arXiv-full-text-chunked
Dataset Card for Dataset Name
This dataset card aims to be a base template for new datasets.
Dataset Details
Dataset Description
Curated by: [More Information Needed]
Funded by [optional]: [More Information Needed]
Shared by [optional]: [More Information Needed]
Language(s) (NLP): [More Information Needed]
License: [More Information Needed]
Dataset Sources [optional]
Repository: [More Information Needed]
Paper [optional]:… See the full description on the dataset page: https://huggingface.co/datasets/amrachraf/arXiv-full-text-chunked.python-text-training-instruct-ai
Python Copilot Instructions on How to Code using Alpaca and Yaml
Training and test datasets for building coding multimodal models that understand how to use the open source GitHub projects for the Agora Open Source AI Research Lab:
Agora GitHub Organization
Agora Hugging Face
This dataset is the 2024-02-03 update for the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details… See the full description on the dataset page: https://huggingface.co/datasets/DevShubham/python-text-training-instruct-ai.ru-text-corpus
Description
798k deduplicated Russian documents (1.6B tokens) from FineWeb-2 (rus_Cyrl), ru-StackOverflow, Pikabu, Habr, Russian Wikipedia and news. Markdown- and code-bearing (StackOverflow/Habr/Pikabu are text_markdown). Filtered to >=200 chars and >=30% Cyrillic. Columns: text, src.
Derived dataset. Source material retains its original per-item licence (see source/repo columns); treat as other / mixed. Provided as-is.
Usage
from datasets import load_dataset… See the full description on the dataset page: https://huggingface.co/datasets/PotatoHD/ru-text-corpus.persian-text-corpus
Persian Corpus (Merged)
Dataset Summary
Persian Corpus (Merged) is a large-scale, Persian corpus meticulously aggregated from multiple high-quality Persian datasets available on the Hugging Face Hub. Designed to advance Persian NLP research and applications, this corpus consolidates diverse textual sources into a single resource, providing researchers and developers with a robust foundation for training and evaluating language models.
Why Use This… See the full description on the dataset page: https://huggingface.co/datasets/PersianML/persian-text-corpus.spider-text-to-sql
Spider Text-to-SQL with LLM-Judge Labels
This dataset extends Spider 1.0 with SQL predictions from gpt-5.4-mini and two correctness labels per example: a hybrid ground truth label and an LLM judge label from gpt-5.4.
Files
File
Description
spider_dataset.parquet
Full dataset with predictions and labels
scripts/
Reproduction scripts (see below)
Dataset statistics
Source: Spider 1.0 training split (train_spider.json)
Databases: the… See the full description on the dataset page: https://huggingface.co/datasets/Glide-py/spider-text-to-sql.judaic-texts-corpus
Judaic Texts Corpus
Dataset Summary
Judaic Texts Corpus is a machine-readable Hebrew and Aramaic corpus of Judaic
texts derived from the Otzaria library release archives. It is intended for
language-model training, retrieval, search, digital humanities research, and
other NLP workflows that need structured access to rabbinic and traditional
Jewish texts.
The current dataset build is produced from the official
Otzaria/otzaria-library release
assets, which package… See the full description on the dataset page: https://huggingface.co/datasets/NHLOCAL/judaic-texts-corpus.text2cypher-2024v1
Neo4j-Text2Cypher (2024) Dataset
The Neo4j-Text2Cypher (2024) Dataset brings together instances from publicly available datasets,
cleaning and organizing them for smoother use. Each entry includes a “question, schema, cypher” triplet at minimum,
with a total of 44,387 instances — 39,554 for training and 4,833 for testing.
An overview of the dataset is shared at Link
Have ideas or insights? Contact us: Neo4j/Team-GenAI
Fields
Fields and their descriptions are as… See the full description on the dataset page: https://huggingface.co/datasets/neo4j/text2cypher-2024v1.cosmopedia-v2-textbook-and-howto-8.3m
Cosmopedia V2 Textbook and WikiHow Dataset 8.3M
This dataset is derived from the HuggingFaceTB Smollm-Corpus with
a specific focus on the Cosmopedia V2 subset.
It contains only entries that are categorized as either textbook, textbook_unconditionned_topic or WikiHow types.
Overview
The Cosmopedia Textbook and WikiHow Dataset is a collection of rows filtered from the original Smollm-Corpus dataset. This dataset is tailored for
researchers and developers who require… See the full description on the dataset page: https://huggingface.co/datasets/schuler/cosmopedia-v2-textbook-and-howto-8.3m.python_text2code
Dataset Card for Python-Text2Code
This dataset supports the EACL paper Text-to-Code Generation with Modality-relative Pre-training
Repository: https://github.com/huawei-noah/noah-research/tree/master/NLP/text2code_mrpt
Point of Contact: Fenia Christopoulou, Gerasimos Lampouras
Dataset Description
The data were crawled from existing, public repositories from GitHub before May 2021 and were meant to be used for
additional model training for the task of Code Synthesis… See the full description on the dataset page: https://huggingface.co/datasets/huawei-noah/python_text2code.task679_hope_edi_english_text_classification
Dataset Card for Natural Instructions (https://github.com/allenai/natural-instructions) Task: task679_hope_edi_english_text_classification
Additional Information
Citation Information
The following paper introduces the corpus in detail. If you use the corpus in published work, please cite it:
@misc{wang2022supernaturalinstructionsgeneralizationdeclarativeinstructions,
title={Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP… See the full description on the dataset page: https://huggingface.co/datasets/Lots-of-LoRAs/task679_hope_edi_english_text_classification.exp01-eeg-to-text-sentences
Exp01 — Sentence-level EEG-to-text training data (unified)
This is a private working corpus for experiment 1 (fine-tuning EEG / time-series
foundation models on EEG-to-English-text). It bundles several public EEG-while-reading
datasets into a single, raw-lossless parquet schema where one row = one sentence read by
one participant.
⚠️ License: Per-source licenses are preserved verbatim in each row's license
column and source_url. Do not re-distribute publicly without re-checking the… See the full description on the dataset page: https://huggingface.co/datasets/tankalapavankalyan/exp01-eeg-to-text-sentences.