Tellurio/PubMed-MultiLabel-MeSH

PubMed MultiLabel Text Classification (MeSH)

A dataset of 50,000 PubMed biomedical articles, each manually annotated by domain experts with MeSH (Medical Subject Headings) labels. With 21,918 unique labels and a mean of ~12.7 labels per document, this is a densely-labeled extreme multi-label classification benchmark.

Dataset Description

Label distribution

MeSH Root Categories

Each label belongs to one or more MeSH root categories. The dataset includes binary indicator columns for the 14 root categories:

Fields

Additional files

• —`label_vocab.json` — ordered list of all 21,918 MeSH label names (index = label ID)
• —`label_metadata.jsonl` — full label metadata including MeSH tree IDs and root categories for hierarchical classification research

Splits

An 80/20 random split with seed 42 (no predefined split exists in the original data).

Usage

from datasets import load_dataset

ds = load_dataset("Tellurio/PubMed-MultiLabel-MeSH")

example = ds["train"][0]
print(example["title"])
print(example["label_names"])  # e.g. ["Humans", "Female", "DNA Probes, HPV", ...]
print(example["label_ids"])    # e.g. [5, 2, 0, ...]
print(example["mesh_roots"])   # e.g. {"A": 0, "B": 1, "C": 1, ...}

Loading label metadata for hierarchical / zero-shot approaches

Each of the 21,918 MeSH labels has associated tree IDs and root categories stored in label_metadata.jsonl.

import json
from huggingface_hub import hf_hub_download

path = hf_hub_download(
    repo_id="Tellurio/PubMed-MultiLabel-MeSH",
    filename="label_metadata.jsonl",
    repo_type="dataset",
)

labels = []
with open(path) as f:
    for line in f:
        labels.append(json.loads(line))

# Example label entry
print(labels[0])
# {"id": 0, "label": "DNA Probes, HPV", "mesh_tree_ids": ["D13.444...", ...], "mesh_roots": ["Chemicals and Drugs [D]"]}

Source

Originally from Kaggle: PubMed MultiLabel Text Classification Dataset MeSH by Owais Ahmad.

Citation

@misc{pubmed_multilabel_mesh,
  author = {Owais Ahmad},
  title = {PubMed MultiLabel Text Classification Dataset MeSH},
  year = {2022},
  publisher = {Kaggle},
  url = {https://www.kaggle.com/datasets/owaiskhan9654/pubmed-multilabel-text-classification}
}

License

CC0: Public Domain