datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
openai-tldr-summarisation-preferences
Human feedback data
This is the version of the dataset used in https://arxiv.org/abs/2310.06452.
If starting a new project we would recommend using https://huggingface.co/datasets/openai/summarize_from_feedback.
See https://github.com/openai/summarize-from-feedback for original details of the dataset.
Here the data is formatted to enable huggingface transformers sequence classification models to be trained as reward functions.
scientific_lay_summarisation-plos-norm
scientific_lay_summarisation - PLOS - normalized
This dataset is a modified version of tomasg25/scientific_lay_summarization and contains scientific lay summaries that have been preprocessed with this code. The preprocessing includes fixing punctuation and whitespace problems, and calculating the token length of each text sample using a tokenizer from the T5 model.
Original dataset details:
Repository: https://github.com/TGoldsack1/Corpora_for_Lay_Summarisation
Paper: Making… See the full description on the dataset page: https://huggingface.co/datasets/pszemraj/scientific_lay_summarisation-plos-norm.DPO-tldr-summarisation-preferences
Dataset Card for DPO-tldr-summarisation-preferences
Reformatted from openai/summarize_from_feedback dataset.
The LION-series are trained using an empirically optimized pipeline that consists of three stages: SFT, DPO, and online preference learning (online DPO). We find simple techniques such as sequence packing, loss masking in SFT, increasing the preference dataset size in DPO, and online DPO training can significantly improve the performance of language models. Our best models… See the full description on the dataset page: https://huggingface.co/datasets/Columbia-NLP/DPO-tldr-summarisation-preferences.scientific_lay_summarisation-elife-norm
scientific_lay_summarisation - elife - normalized
This is the "elife" split. For more words, refer to the PLOS split README
Contents
load with datasets:
from datasets import load_dataset
# If the dataset is gated/private, make sure you have run huggingface-cli login
dataset = load_dataset("pszemraj/scientific_lay_summarisation-elife-norm")
dataset
Output:
DatasetDict({
train: Dataset({
features: ['article', 'summary', 'section_headings', 'keywords', 'year'… See the full description on the dataset page: https://huggingface.co/datasets/pszemraj/scientific_lay_summarisation-elife-norm.pmc-medical-summarisationLT_Summarisation_Corpus
Lithuanian Summarisation Corpus
Description
Lithuanian texts paired with human-written abstractive and extractive summaries, spanning four subject domains — information technology, law, medicine, and media. The corpus is designed for training and evaluating summarisation systems on Lithuanian-language content, covering both technical and general registers.
Dataset summary
Types of Dataset: CSV, JSON, XML
Number of CSV files: 8 (4 training, 4… See the full description on the dataset page: https://huggingface.co/datasets/VytautoDidziojoUniversitetas/LT_Summarisation_Corpus.ekacare_medical_history_summarisation
Medical History Summarization Dataset
This dataset contains 58 diverse medical cases from EkaCare's internal medical team and the company's employees and their family members. Each case provides a complete view of a patient's medical journey, including vital sign trends and historical health context.
The dataset has been designed to evaluate the ability to summarize complex medical histories and extract clinically relevant insights. For details read this blog.
Dataset… See the full description on the dataset page: https://huggingface.co/datasets/ekacare/ekacare_medical_history_summarisation.tldr-summarisation-preferences_reformatted
Dataset Card for when2rl/tldr-summarisation-preferences_reformatted
Reformatted from UCL-DARK/openai-tldr-summarisation-preferences to be consistent with all other datasets in this org. Note that since the original dataset does not have score ratings, we used "10" for ALL chosen response, and "1" for ALL rejected response as dummy scores.
Dataset Details
Dataset Description
Curated by: [More Information Needed]
Funded by [optional]: [More Information… See the full description on the dataset page: https://huggingface.co/datasets/when2rl/tldr-summarisation-preferences_reformatted.judgement-summarisation-llama-2toxic-dialogue-summarisation
Dataset Card for Dataset Name
This dataset card aims to be a base template for new datasets. It has been generated using this raw template.
Dataset Details
Dataset Description
Curated by: [Aditya Singh]
Language(s) (NLP): [English]
License: [MIT]
Dataset Sources [Grok3]
Repository: [More Information Needed]
Demo [optional]: [More Information Needed]
Uses
Direct Use
[More Information Needed]
Out-of-Scope… See the full description on the dataset page: https://huggingface.co/datasets/majorSeaweed/toxic-dialogue-summarisation.scientific-lay-summarisationSummarisation_datasetwhatsapp-summarisation-datasetzephyr_text_summarisation_500Malayalam_Summarisationsummarisationsalt-summarisationsummarisation