datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
Scientific-Summaries
Scientific Summaries
22 million LLM-generated structured summaries of scientific papers, enriched with OpenAlex scholarly metadata. Each paper has an 18-field structured summary covering methodology, key results, claims, limitations, and more. This public dataset includes full paper text for ~5.3 million papers where open-access status has been confirmed -- either through OpenAlex metadata or because the paper originates from a permissively licensed source such as the arXiv preprint… See the full description on the dataset page: https://huggingface.co/datasets/laion/Scientific-Summaries.summeval
SummEvalSummarization.v2
An MTEB dataset
Massive Text Embedding Benchmark
News Article Summary Semantic Similarity Estimation. This version fixes a bug in the evaluation script that caused the main score to be computed incorrectly.
Task category
t2t
Domains
News, Written
Reference
https://github.com/Yale-LILY/SummEval
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task =… See the full description on the dataset page: https://huggingface.co/datasets/mteb/summeval.arxiv-summarization
Arxiv dataset for summarization
Dataset for summarization of long documents.Adapted from this repo.Note that original data are pre-tokenized so this dataset returns " ".join(text) and add "\n" for paragraphs. This dataset is compatible with the run_summarization.py script from Transformers if you add this line to the summarization_name_mapping variable:
"ccdv/arxiv-summarization": ("article", "abstract")
Data Fields
id: paper id
article: a string containing the body of… See the full description on the dataset page: https://huggingface.co/datasets/ccdv/arxiv-summarization.pubmed-summarization
PubMed dataset for summarization
Dataset for summarization of long documents.Adapted from this repo.Note that original data are pre-tokenized so this dataset returns " ".join(text) and add "\n" for paragraphs. This dataset is compatible with the run_summarization.py script from Transformers if you add this line to the summarization_name_mapping variable:
"ccdv/pubmed-summarization": ("article", "abstract")
Data Fields
id: paper id
article: a string containing the body… See the full description on the dataset page: https://huggingface.co/datasets/ccdv/pubmed-summarization.openai_summarize_tldr
Dataset Card for "openai_summarize_tldr"
More Information needed
openai_summarize_comparisonsKMMLU-Summarized-Chain_of_Thought
Dataset Card for Condensed Chain-of-Thought KMMLU Dataset
This dataset card provides detailed information about the condensed KMMLU dataset. The dataset has been summarized using Upstage's LLM: Solar-Pro to condense the original KMMLU training and development data while preserving its quality and usability. Additionally, a new column, 'chain_of_thought', has been introduced to align with the reasoning approach outlined in the paper "Chain-of-Thought Prompting Elicits Reasoning in… See the full description on the dataset page: https://huggingface.co/datasets/SabaPivot/KMMLU-Summarized-Chain_of_Thought.govreport-summarization
GovReport dataset for summarization
Dataset for summarization of long documents.Adapted from this repo and this paperThis dataset is compatible with the run_summarization.py script from Transformers if you add this line to the summarization_name_mapping variable:
"ccdv/govreport-summarization": ("report", "summary")
Data Fields
id: paper id
report: a string containing the body of the reportsummary: a string containing the summary of the report
Data Splits… See the full description on the dataset page: https://huggingface.co/datasets/ccdv/govreport-summarization.SUMM-RENote: if the data viewer is not working, use the "example" subset.
SUMM-RE
The SUMM-RE dataset is a collection of transcripts of French conversations, aligned with the audio signal.
It is a corpus of meeting-style conversations in French created for the purpose of the SUMM-RE project (ANR-20-CE23-0017).
The full dataset is described in Hunter et al. (2024): "SUMM-RE: A corpus of French meeting-style conversations".
Created by: Recording and manual correction of the corpus was… See the full description on the dataset page: https://huggingface.co/datasets/linagora/SUMM-RE.summarize_from_feedbackSummarize from Feedback contains the human feedback data released by the "Learning to summarize from human feedback" paper.NLG-Abstractive-Summarization
SEA Abstractive Summarization
SEA Abstractive Summarization evaluates a model's ability to read a document, identify the key points within, and summarize them into a coherent and fluent text while paraphrasing the document. It is sampled from XL-Sum for Indonesian, Tamil, Thai, and Vietnamese.
Supported Tasks and Leaderboards
SEA Abstractive Summarization is designed for evaluating chat or instruction-tuned large language models (LLMs). It is part of the SEA-HELM… See the full description on the dataset page: https://huggingface.co/datasets/aisingapore/NLG-Abstractive-Summarization.news-summary
Dataset Card for "news-summary"
Dataset Summary
Officially it was supposed to be used for classification but, can you use this data set to summarize news articles?
Languages
english
Citation Information
Acknowledgements
Ahmed H, Traore I, Saad S. “Detecting opinion spams and fake news using text classification”, Journal of Security and Privacy, Volume 1, Issue 1, Wiley, January/February 2018.
Ahmed H, Traore I, Saad S. (2017) “Detection of Online… See the full description on the dataset page: https://huggingface.co/datasets/argilla/news-summary.cmu-book-summarieshf_paper_summary
Paper Espresso Dataset
This dataset repository contains structured metadata, summaries, and topical analysis for trending AI research papers, as presented in the paper Paper Espresso: From Paper Overload to Research Insight.
Paper Espresso is an open-source platform designed to automatically discover, summarize, and analyze trending research papers from arXiv. The system uses large language models (LLMs) to generate structured summaries, topical labels, and keywords. Over 35… See the full description on the dataset page: https://huggingface.co/datasets/Elfsong/hf_paper_summary.t3_data_V2wavepulse-radio-summarized-transcripts
WavePulse Radio Summarized Transcripts
Dataset Summary
WavePulse Radio Summarized Transcripts is a large-scale dataset containing summarized transcripts from 396 radio stations across the United States, collected between June 26, 2024, and October 3, 2024. The dataset comprises approximately 1.5 million summaries derived from 485,090 hours of radio broadcasts, primarily covering news, talk shows, and political discussions.
The raw version of the transcripts is available… See the full description on the dataset page: https://huggingface.co/datasets/nyu-dice-lab/wavepulse-radio-summarized-transcripts.legal_summarization
LegalSummarization
An MTEB dataset
Massive Text Embedding Benchmark
The dataset consistes of 439 pairs of contracts and their summarizations from https://tldrlegal.com and https://tosdr.org/.
Task category
t2t
Domains
Legal, Written
Reference
https://github.com/lauramanor/legal_summarization
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task =… See the full description on the dataset page: https://huggingface.co/datasets/mteb/legal_summarization.asta-summary-citation-counts
Dataset Summary
This dataset tracks which scientific papers are most often cited by Asta, an agentic research platform that uses retrieval-augmented generation (RAG) to answer scientific questions. Each record is a paper cited by Asta's Summarize Literature tool, ranked by the number of times the system cited that paper. Across more than 113,000 user queries, we track 4M citations to over 2M distinct papers. By making this data public, we aim to create a transparent, trackable… See the full description on the dataset page: https://huggingface.co/datasets/allenai/asta-summary-citation-counts.summ_dst3_dataset_main_npzarxiv_deep_learning_python_research_code_functions_summaries
Dataset Card for "AlgorithmicResearchGroup/arxiv_deep_learning_python_research_code_functions_summaries"
Dataset Description
https://huggingface.co/datasets/AlgorithmicResearchGroup/arxiv_deep_learning_python_research_code_functions_summaries
Dataset Summary
AlgorithmicResearchGroup/arxiv_deep_learning_python_research_code_functions_summaries contains summaries for every python function and class extracted from source code files referenced in ArXiv papers. The… See the full description on the dataset page: https://huggingface.co/datasets/AlgorithmicResearchGroup/arxiv_deep_learning_python_research_code_functions_summaries.image-descr-v2nemotron-cc-v21-Parsed-QA4-Summarization-Qwen3-1.7Bsummarize_from_feedback_smallmedical-dialogue-to-soap-summary
Dataset Card for Synthetic Medical Dialogues and SOAP Summaries
Dataset Description
Abstract
This dataset consists of 10,000 synthetic dialogues between a patient and clinician, created using the GPT-4 dataset from NoteChat, based on PubMed Central (PMC) case-reports. Accompanying these dialogues are SOAP summaries generated through GPT-4. The dataset is split into 9250 training, 500 validation, and 250 test entries, each containing a dialogue column, a SOAP… See the full description on the dataset page: https://huggingface.co/datasets/omi-health/medical-dialogue-to-soap-summary.wiki-movie-plots-with-summaries
Dataset Card for Wikipedia Movie Plots with AI Plot Summaries
Dataset Summary
Context
Wikipedia Movies Plots dataset by JustinR ( https://www.kaggle.com/jrobischon/wikipedia-movie-plots )
Content
Everything is the same as in https://www.kaggle.com/jrobischon/wikipedia-movie-plots
Acknowledgements
Please, go upvote https://www.kaggle.com/jrobischon/wikipedia-movie-plots dataset, since this is 100% based on that.
Supported Tasks and… See the full description on the dataset page: https://huggingface.co/datasets/vishnupriyavr/wiki-movie-plots-with-summaries.speech-translation-and-summarization
English-Centric Multilingual Audio Dataset
This dataset contains generated article and summary audio for English-centric multilingual directions.
Each direction folder contains metadata JSONL files and corresponding audio files for few_shot and test splits.
Included directions
amharic_english / english_amharic
arabic_english / english_arabic
bengali_english / english_bengali
chinese_simplified_english / english_chinese_simplified
english_english
french_english /… See the full description on the dataset page: https://huggingface.co/datasets/McGill-NLP/speech-translation-and-summarization.flawed-summ-evalslatin-summarizer-dataset
✨ LatinSummarizer Dataset ✨
Note: If Dataset Viewer is not available, see samples of dataset for samples from the dataset.The LatinSummarizer Dataset is a comprehensive collection of Latin texts designed to support natural language processing research for a low-resource language. It provides parallel data for various tasks, including translation (Latin-to-English) and summarization (extractive and abstractive).
This dataset was created for a… See the full description on the dataset page: https://huggingface.co/datasets/LatinNLP/latin-summarizer-dataset.Generated_summaries