CoolFace
Datasetpublic

jimmyzxj/drosophila-literature-corpus

Drosophila Literature Corpus A corpus of full-text scientific articles from PubMed Central related to Drosophila melanogaster gene research. Dataset Description This corpus contains ~17,000 full-text scientific articles downloaded from the PubMed Central Open Access Subset via the BioC-PMC API. The articles are associated with genes that have expert-curated summaries in FlyBase. Usage from datasets import load_dataset corpus =… See the full description on the dataset page: https://huggingface.co/datasets/jimmyzxj/drosophila-literature-corpus.

sourceHugging Facecc-by-4.0updated 9mo agoView on Hugging Face
0likes29downloads
Dataset Card

Drosophila Literature Corpus

A corpus of full-text scientific articles from PubMed Central related to Drosophila melanogaster gene research.

Dataset Description

This corpus contains ~17,000 full-text scientific articles downloaded from the PubMed Central Open Access Subset via the BioC-PMC API. The articles are associated with genes that have expert-curated summaries in FlyBase.

Usage

python
from datasets import load_dataset

corpus = load_dataset("TODO/drosophila-literature-corpus")

# Access a document
doc = corpus["train"][0]
print(doc["pmcid"])
print(doc["title"])
print(doc["abstract"])
print(doc["sections"])  # Dict with keys: INTRO, METHODS, RESULTS, DISCUSS, CONCL

Dataset Structure

Each record contains:

FieldTypeDescription
pmcidstringPubMed Central ID
titlestringArticle title
abstractstringArticle abstract
sectionsdictMapping of section type to list of paragraphs

Section Types

  • —INTRO - Introduction
  • —METHODS - Methods/Materials
  • —RESULTS - Results
  • —DISCUSS - Discussion
  • —CONCL - Conclusion

Dataset Statistics

  • —Total documents: 16,898
  • —Source: PubMed Central Open Access Subset
  • —Format: BioC JSON (converted to JSONL)

Source

Articles were retrieved using the BioC-PMC API:

https://www.ncbi.nlm.nih.gov/research/bionlp/RESTful/pmcoa.cgi/BioC_json/{pmcid}/unicode

Related

This corpus is part of the Drosophila Agent Benchmark, which evaluates AI agents on literature-based gene annotation tasks.

License

The articles in this corpus are from the PubMed Central Open Access Subset and are available under various Creative Commons licenses. Please refer to individual article licenses for specific terms.