CoolFace
Datasetpublic

bluuebunny/crossref_metadata_2025_split

Dataset Overview This dataset contains bibliographic metadata from the public Crossref snapshot released in 2025. It provides core fields for scholarly documents, including DOI, title, abstract, authorship, publication month and year, and URLs. The entire public dump (~196.94 GB) was filtered and extracted into a parquet format for efficient loading and querying. Total size: 196.94 GB (parquet files) Number of records: 34,308,730 Use this dataset for large-scale text mining… See the full description on the dataset page: https://huggingface.co/datasets/bluuebunny/crossref_metadata_2025_split.

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face
0likes161downloads
Dataset Card

Dataset Overview

This dataset contains bibliographic metadata from the public Crossref snapshot released in 2025. It provides core fields for scholarly documents, including DOI, title, abstract, authorship, publication month and year, and URLs. The entire public dump (\~196.94 GB) was filtered and extracted into a parquet format for efficient loading and querying.

  • —Total size: 196.94 GB (parquet files)
  • —Number of records: 34,308,730

Use this dataset for large-scale text mining, bibliometric analyses, metadata enrichment, and building citation-aware tools.

Dataset Features

Each record in the dataset contains the following fields:

FieldTypeDescription
doistringDigital Object Identifier of the publication.
titlestringTitle of the scholarly work.
abstractstringAbstract text (when available).
authorlistList of author names or structured author metadata.
monthintPublication month (1–12).
yearintPublication year (e.g., 2024, 2025).
urlstringURL pointing to the publication page or DOI resolver.

Dataset Structure

The dataset is provided in Apache Parquet format. Parquet allows for efficient columnar storage and supports schema evolution. Each parquet file chunk contains the complete schema as described above.

root
 |-- doi: string (nullable = true)
 |-- title: string (nullable = true)
 |-- abstract: string (nullable = true)
 |-- author: array (nullable = true)
 |    |-- element: string (nullable = true)
 |-- month: int (nullable = true)
 |-- year: int (nullable = true)
 |-- url: string (nullable = true)

Dataset Splits

This dataset does not come with predefined splits. Users can split based on publication year, subject areas, or random sampling as per their experimentation needs.

Dataset Creation

Source

  • —Access method: Downloaded the public JSON dump from Academic Torrents. See

Processing

  1. 1.Extraction: Parsed the Crossref dump to extract relevant fields (DOI, title, abstract, authors, month, year, URL).
  2. 2.Transformation: Normalized fields; authors consolidated into a list of names.
  3. 3.Serialization: Saved the resulting table in Parquet format for columnar efficiency.
  4. 4.Storage: Uploaded parquet files to Hugging Face Datasets with corresponding metadata.

Code for dataset processing and card generation is available at:

https://github.com/mitanshu7/PaperMatch_crossref

Usage

python
from datasets import load_dataset

dataset = load_dataset(
    "bluuebunny/crossref_metadata_2025",
    streaming=True,
    split='train'
)

# Inspect a few records
print(dataset[0])

# Filter by year
subset_2025 = dataset.filter(lambda x: x["year"] == 2025)
print(f"Records published in 2025: {len(subset_2025)}")

Citation

If you use this dataset in your research, please cite the Crossref public data file:

@misc{crossref2025,
  title        = {{Crossref} Public Data File 2025},
  author       = {{Crossref}},
  year         = 2025,
  howpublished = {\url{https://www.crossref.org/blog/2025-public-data-file-now-available/}},
}

Contact