datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
arxiv-abstracts-largeThe arXiv Dataset is a comprehensive knowledge repository of 1.7 million scholarly articles drawn from the vast domains of physics, computer science, statistics, electrical engineering, quantitative biology, and economics among others. It provides open access to vital features such as article titles, authors, categories, abstracts, full text PDFs, and more. The dataset offers immense depth, allowing for exploration into various subdisciplines and interconnections between them. It serves as a… See the full description on the dataset page: https://huggingface.co/datasets/UniverseTBD/arxiv-abstracts-large.arxiv_abstracts
ArXiv Abstracts
Description
Each paper uploaded to ArXiv includes structured metadata fields, including an abstract summarizing the paper’s findings and contributions.
According to ArXiv’s licensing policy, the metadata for any paper submitted to ArXiv is distributed under the CC0 license, regardless of the license of the paper itself.
Thus, this dataset contains the abstract for every paper submitted to ArXiv through late 2024.
We source the abstracts from ArXiv’s API… See the full description on the dataset page: https://huggingface.co/datasets/common-pile/arxiv_abstracts.arxiv_abstracts_filtered
ArXiv Abstracts
Description
Each paper uploaded to ArXiv includes structured metadata fields, including an abstract summarizing the paper’s findings and contributions.
According to ArXiv’s licensing policy, the metadata for any paper submitted to ArXiv is distributed under the CC0 license, regardless of the license of the paper itself.
Thus, this dataset contains the abstract for every paper submitted to ArXiv through late 2024.
We source the abstracts from ArXiv’s API… See the full description on the dataset page: https://huggingface.co/datasets/common-pile/arxiv_abstracts_filtered.the-pile-pubmed-abstracts-refined-by-data-juicer
The Pile -- PubMed Abstracts (refined by Data-Juicer)
A refined version of PubMed Abstracts dataset in The Pile by Data-Juicer. Removing some "bad" samples from the original dataset to make it higher-quality.
This dataset is usually used to pretrain a Large Language Model.
Notice: Here is a small subset for previewing. The whole dataset is available here (About 24G).
Dataset Information
Number of samples: 371,331 (Keep ~99.55% from the original dataset)… See the full description on the dataset page: https://huggingface.co/datasets/datajuicer/the-pile-pubmed-abstracts-refined-by-data-juicer.3M_Academic_Papers_Titles_and_Abstracts
Comprehensive Academic Papers Dataset: 3M+ Research Paper Titles and Abstracts
📋 Overview
This dataset is a comprehensive collection of over 3 million research paper titles and abstracts, curated and consolidated from multiple high-quality academic sources. The dataset provides a unified, clean, and standardized format for researchers, data scientists, and machine learning practitioners working on natural language processing, academic research analysis, and knowledge… See the full description on the dataset page: https://huggingface.co/datasets/beta3/3M_Academic_Papers_Titles_and_Abstracts.arxiv-abstracts-2004
ArXiv Abstracts 2004
Original Dataset: common-pile/arxiv_abstracts
ArXiv-Abstracts-2004 is a filtered collection of abstracts from the Common-Pile ArXiv dataset containing works created on or before 2004.
Stats
Size (MB)
Lines
351MB
303,761
Note: The lines, in the .jsonl file, are ordered from oldest to newest.
Notice
We do not claim ownership of or credit for any prior work done by the Common-Pile team. This dataset is only a… See the full description on the dataset page: https://huggingface.co/datasets/fromziro/arxiv-abstracts-2004.bilingual-abstracts-corpus
ÚFAL Bilingual Abstracts Corpus
This is a parallel (bilingual) corpus of Czech and mostly English abstracts of scientific papers and presentations published by authors from the Institute of Formal and Applied Linguistics, Charles University in Prague.
For each publication record, the authors are obliged to provide both the original abstract (in Czech or English), and its translation (English or Czech) in the internal Biblio system.
The data was filtered for duplicates and missing… See the full description on the dataset page: https://huggingface.co/datasets/ufal/bilingual-abstracts-corpus.econ_paper_abstracts
Dataset Card for Economics Paper Dataset
Dataset Summary
The Economics Research Paper Dataset was designed to support the development of the LLaMA-2-Econ models, with a focus on Title Generation, Abstract Classification, and Question & Answer (Q&A) tasks. It comprises abstracts and titles of economics research papers, along with synthetic Q&A pairs derived from the abstracts, to facilitate training of large language models for economics-specific applications.… See the full description on the dataset page: https://huggingface.co/datasets/onurkeles/econ_paper_abstracts.pubmed-bioinformatics-abstracts_all_years
PubMed Bioinformatics Abstracts — All Years
A curated dataset of bioinformatics research abstracts fetched from PubMed via the NCBI Entrez API, covering publications from 1990 through 2026. Designed for LLM fine-tuning, biomedical NLP, and scientific text analysis.
Curated by: Dr. Yash M Gupta
Pipeline: yashmgupta/pubmed-bioinformatics-abstracts
Intended Use
Fine-tuning large language models (LLMs) on biomedical / bioinformatics domain text.
Data Fields… See the full description on the dataset page: https://huggingface.co/datasets/yashm/pubmed-bioinformatics-abstracts_all_years.pubmed-bioinformatics-abstracts
PubMed Bioinformatics Abstracts (2020–2026)
A curated collection of 174,154 bioinformatics journal article abstracts from PubMed,
covering publications from 2020 to 2026.
Curated by: Dr. Yash M Gupta
Intended Use
Fine-tuning large language models (LLMs) on biomedical / bioinformatics domain text.
Dataset Structure
Column
Type
Description
pmid
string
PubMed ID
title
string
Article title
abstract
string
Full abstract text
year
string
Publication… See the full description on the dataset page: https://huggingface.co/datasets/yashm/pubmed-bioinformatics-abstracts.