christian-hoang-04/BiomedKeyRefine

<p align="center"> <img src="https://huggingface.co/datasets/christian-hoang-04/BiomedKeyRefine/resolve/main/BiomedKeyRefine.png" width="800"> </p>

BiomedKeyRefine: Refined Biomedical Keyword Extraction Benchmark

Version 1.0 Last updated: February 2026 Author: Christian Hoang (@christhoang04) License: MIT

🌟 Dataset Summary

BiomedKeyRefine is a high-quality, refined subset of the PubMedAKE benchmark (CIKM 2022), specifically curated for biomedical keyword extraction research.

The original PubMedAKE contains 843k+ articles with author-assigned keywords (extractive + abstractive). This version:

• —Filters for quality (≥5 keywords total, abstract ≥200 chars, removed noisy entries >40% stop words).
• —Stratifies into multi-label sub-domains (genomics, clinicaltrials, drugdiscovery, epidemiology).
• —Enables evaluation of domain-specific knowledge impact on keyword extraction (correlation with external benchmarks like GUE, CSEDB).
• —Supports LLM-as-a-judge with non-expert calibration.

Ideal for:

• —Keyword extraction (extractive/abstractive)
• —Domain adaptation & knowledge probing
• —LLM evaluation in biomedical text mining
• —Correlation between domain knowledge and extraction performance

📈 Dataset Statistics

Sub-domain Distribution (multi-label occurrences):

• —epidemiology: ~1,741
• —clinical_trials: ~1,467
• —genomics: ~1,414
• —drug_discovery: ~740

(Note: Multi-label means some articles belong to multiple domains, reflecting real interdisciplinary nature of biomedical research.)

🛠️ Supported Tasks and Leaderboards

• —Keyword Extraction (extractive & abstractive)
• —Domain Classification (multi-label from sub_domains)
• —Knowledge Correlation (link domain knowledge benchmarks to extraction F1)
• —LLM-as-a-Judge Evaluation (with non-expert calibration)

No official leaderboard yet – community contributions welcome!

🌍 Languages

• —Primarily English (all abstracts, titles, keywords from PubMed Central English subset).

📂 Dataset Structure

Data Instances

Each row is one biomedical article with:

• —Metadata (PMC_ID, title)
• —Content (abstract)
• —Ground truth (keywordsin: extractive, keywordsnot_in: abstractive)
• —Labels (subdomains: multi-label string, e.g., "genomics, drugdiscovery")

Example (from clinical_trials):

{
  "keywords_in": ["Clinical trials", "COVID-19", "Estimands", "Missing data", "Pandemic"],
  "keywords_not_in": ["Statistical strategy", "Supportive analyses"],
  "title": "Statistical Issues and Recommendations for Clinical Trials Conducted During the COVID-19 Pandemic",
  "abstract": "The COVID-19 pandemic has had and continues to have major impacts on planned and ongoing clinical trials...",
  "PMC_ID": "PMC8011486",
  "sub_domains": "clinical_trials, epidemiology"
}

📂 Data Fields

Data Splits

Split Type 1: Standard Train/Validation/Test Split (70/15/15, stratified by primary domain)

• —train: 3,500 articles (70%) – for fine-tuning or training judge
• —validation: 750 articles (15%) – for hyperparameter tuning & calibration
• —test: 750 articles (15%) – for final evaluation & baseline comparison

Split Type 2: Sub-domain Split (multi-label, with overlap between domains)

Separate files for each primary sub-domain (easy to load domain-specific data):

• —subdomain_genomics.csv (~1,414 articles)
• —subdomainclinicaltrials.csv (~1,467 articles)
• —subdomaindrugdiscovery.csv (~740 articles)
• —subdomain_epidemiology.csv (~1,741 articles)

Benefit: Convenient for domain-specific testing (e.g., run only on genomics to correlate with GUE benchmark).

All splits are stratified to preserve balance across primary domains.

All splits are stratified by primary domain (first sub-domain in multi-label list) to maintain balance.

🛠️ Dataset Creation

Curation Rationale

Refined from PubMedAKE to create a smaller, higher-quality subset focused on:

• —Clean ground truth (≥5 keywords, long abstract, low noise).
• —Sub-domain stratification for domain knowledge studies.
• —Feasibility for LLM evaluation and fine-tuning on consumer hardware.

Source Data

• —Original: PubMed Open Access Subset (PMC-OA) via PubMedAKE (Zenodo DOI: 10.5281/zenodo.6330817)
• —Filtering: ≥5 keywords total, abstract ≥200 chars, removed noisy entries (>40% stop words)

Annotations

• —Ground truth: Author-assigned keywords (extractive + abstractive) from original PubMedAKE.
• —Sub-domains: Inferred multi-label from keywords using domain-specific terms.

Personal and Sensitive Information

No personal or sensitive data (only public PMC articles).

⚠️ Considerations for Using the Data

Social Impact

Improves biomedical literature search, recommendation, and text mining (PubMed, Semantic Scholar, drug discovery RAG).

Limitations

• —Skew toward epidemiology (PubMed OA bias).
• —Author keywords can be subjective or noisy.
• —English-only.
• —Sub-domain inference heuristic (keyword-based) may have minor overlap errors.

Misuse Potential

None foreseen (public scientific abstracts).

🚀 How to Use

from datasets import load_dataset

# Load standard splits
dataset = load_dataset("christian-hoang-04/BiomedKeyRefine")

train_df = dataset["train"].to_pandas()
print(train_df.head())

# Load sub-domain specific file (download from repo)
genomics = pd.read_csv("https://huggingface.co/datasets/christian-hoang-04/BiomedKeyRefine/resolve/main/subdomains/genomics.csv")

📚 Citation

If you use this dataset in your research, please cite:

@misc{hoang2026biomedkeyrefine,
  author       = {Christian Hoang},
  title        = {BiomedKeyRefine: Refined Biomedical Keyword Extraction Benchmark with Sub-domain Stratification},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/datasets/christian-hoang-04/BiomedKeyRefine}},
}

🔗 References / Related Datasets

• —Original PubMedAKE: https://pmc.ncbi.nlm.nih.gov/articles/PMC9652778/
• —Zenodo source: https://zenodo.org/records/6330817