CoolFace
Datasetpublic

yufan/arxiv-metadata-2020-2026

arXiv Metadata, enriched (2020–2026) Per-paper metadata for 1,517,185 arXiv papers spanning 2020-01 → 2026-09, enriched with abstracts, citation counts, and Semantic Scholar identifiers, and organized as a two-level hierarchy: field of study → year. Unlike a bare title index, every record here carries the abstract, the full author list, citation counts, and the Semantic Scholar corpusId, so you can do retrieval, classification, citation analysis, and corpus building directly… See the full description on the dataset page: https://huggingface.co/datasets/yufan/arxiv-metadata-2020-2026.

sourceHugging Faceodc-byupdated 7d agoView on Hugging Face
0likes1.1kdownloads
Dataset Card

arXiv Metadata, enriched (2020–2026)

Per-paper metadata for 1,517,185 arXiv papers spanning 2020-01 → 2026-09, enriched with abstracts, citation counts, and Semantic Scholar identifiers, and organized as a two-level hierarchy: field of study → year.

Unlike a bare title index, every record here carries the abstract, the full author list, citation counts, and the Semantic Scholar `corpusId`, so you can do retrieval, classification, citation analysis, and corpus building directly from the metadata.

Layout: field → year

Files are stored as `<Field>/<Year>/metadata.jsonl`, one JSON object per line. Fields (the first level) are ordered largest → smallest by paper count:

Field (`config`)2020202120222023202420252026**Total**
Computer_Science68,95573,88377,86693,422117,099140,576116,036687,837
Physics57,50161,08760,94264,90469,90679,59964,325458,264
Mathematics34,24031,84432,30434,72239,11944,01341,060257,302
Medicine4,7296,6426,9716,7417,0675,9721,15639,278
Engineering2,9413,5963,8454,5585,8477,2907,17535,252
Economics1,2801,7981,7982,0412,5273,1422,66515,251
Biology1,3271,2671,1521,4301,7512,3321,70510,964
Materials_Science4,57061553476835195,407
Unknown7776026966005711,4883755,109
Chemistry2805617332134405
Geology3095410101431401
Business2885212377–369
Environmental_Science29039851142359
Psychology2812952441326
Geography205138111–229
Political_Science132142344–159
Sociology108191–21–131
Philosophy5981––1–69
History51101––1–63
Art62–11––10
all178,329181,630185,692208,493244,031284,486234,5241,517,185

(– means that field has no papers harvested for that year.)

Columns

ColumnTypeDescription
arxiv_idstringarXiv identifier, new-style YYMM.NNNNN (e.g. 2512.25075), no version suffix.
corpusIdintSemantic Scholar corpus id (stable internal key).
titlestringPaper title.
authorsstringFull author list, ; -separated display names.
yearintPublication year (matches the folder).
publicationDatestringYYYY-MM-DD when available, else null.
citationCountintCitations known to Semantic Scholar at harvest time.
influentialCitationCountintInfluential-citation subset (Valenzuela et al.).
fieldsOfStudystringPrimary field of study (matches the folder).
abstractstringPaper abstract; may be null when not available.
fetched_atstringUTC ISO-8601 timestamp of when the record was harvested.

Usage

python
from datasets import load_dataset

# One field, all years (each year is a split):
cs = load_dataset("yufan/arxiv-metadata-2020-2026", "Computer_Science")
print(cs)                      # splits: 2020 … 2026
print(cs["2025"][0])

# One field + one year:
phys25 = load_dataset("yufan/arxiv-metadata-2020-2026", "Physics", split="2025")

# Stream the largest split without downloading everything:
it = load_dataset("yufan/arxiv-metadata-2020-2026", "Computer_Science",
                  split="2025", streaming=True)
print(next(iter(it)))

Download a single raw file directly:

python
from huggingface_hub import hf_hub_download
path = hf_hub_download(
    repo_id="yufan/arxiv-metadata-2020-2026",
    filename="Computer_Science/2025/metadata.jsonl",
    repo_type="dataset",
)

Provenance

  • —arXiv — base identifiers and titles, via arXiv's open metadata.
  • —Semantic Scholar — abstracts, author lists, corpusId, fieldsOfStudy, and citation counts, via the Semantic Scholar Academic Graph API.
  • —Date window: 2020-01 through ~July 2026; 2026 is partial.
  • —fieldsOfStudy is Semantic Scholar's primary field; papers it could not place are grouped under `Unknown`.

License & attribution

The enriched fields originate from the Semantic Scholar Academic Graph, released under ODC-BY 1.0 — please attribute Semantic Scholar (and arXiv for the underlying papers) when you use this data. arXiv's own metadata is provided under CC0; see arXiv's Terms of Use.

Citation

bibtex
@misc{arxiv_metadata_enriched_2020_2026,
  title  = {arXiv Metadata, enriched (2020--2026), by field and year},
  author = {yufan},
  year   = {2026},
  url    = {https://huggingface.co/datasets/yufan/arxiv-metadata-2020-2026}
}