almanach/OntoBook

[image]

OntoBook: Ontology-Grounded Synthetic Textbooks for Medical Encoder Pretraining

Dataset Authors

Rian Touchent & Eric de la Clergerie Inria, Sorbonne Université

Overview

OntoBook is a French biomedical pretraining corpus generated from the relational structure of three medical ontologies: CIM-10 FR PMSI for diagnoses, CCAM for medical procedures, and ATC for drugs. Weighted random walks turn ontology graphs into structured sequences of codes and relations. Qwen3-235B-A22B-Instruct-2507 then reformulates each walk into continuous textbook-style medical prose.

Two encoders pretrained on this corpus are available: ModernCamemBERT-bio-v2-base and ModernCamemBERT-bio-v2-large.

The generation prompt requires the model to preserve the source codes, relations, definitions, notes, inclusions, and exclusions while avoiding information absent from the walk. The release retains both representations: the structured source_walk for provenance and the fluent text for language-model pretraining.

Token counts are estimated with almanach/moderncamembert-base from a stratified sample of 15,189 rows and exact character totals for the full release, without per-document special tokens. The structured source_walk field is estimated at approximately 1.36 billion tokens with the same method.

Load the Dataset

Use a current version of datasets (pip install -U datasets) so named splits declared in the dataset card are resolved correctly.

from datasets import load_dataset

# Load one ontology.
dataset = load_dataset("almanach/OntoBook", split="cim10")

# The three available splits are: cim10, ccam, and atc.
print(dataset[0]["text"])

Dataset Structure

Each split uses the same schema.

Walk Types

The CIM-10 configuration contains five targeted walk families: etiologie, diagnostic_differentiel, codage_double, syndrome, and cross_chapter. CCAM and ATC use hierarchie, comparaison, and exploration walks adapted to procedures and therapeutic classes.

Generation Process

1. 1.Medical ontologies are parsed from their RDF/OWL distributions.
2. 2.Weighted random walks traverse hierarchical and semantic relations. Most walks contain 8–20 steps, with longer cross-chapter walks allowed for CIM-10.
3. 3.Qwen3-235B-A22B-Instruct-2507 reformulates the walks with temperature 0, guided JSON decoding, and thinking disabled.
4. 4.The release builder normalizes Unicode and whitespace, standardizes walk-type labels, generates stable identifiers, and writes compressed Parquet shards.

Generation used vLLM with FP8 inference on four NVIDIA H100 GPUs. The complete reformulation stage took approximately 20 hours.

Quality Controls

This release is built from pinned revisions of the three original datasets. All rows were checked for required values, valid token counts, known walk types, and duplicate content. Cleaning removed 356 exact duplicates and 60 ATC rows with an empty label. The final release contains no null fields or duplicate identifiers.

The counts in this card describe the cleaned, reformulated release. They differ from the number of raw ontology walks reported before generation and filtering.

Intended Uses

OntoBook is intended for biomedical language-model pretraining, ontology-aware representation learning, medical coding research, and controlled studies of knowledge-graph verbalization. The source_walk and text pair can also support research on grounded generation and structure-to-text transformation.

Limitations

The corpus is synthetic and may contain generation errors despite constrained prompting. It must not be treated as clinical guidance or used directly for diagnosis, treatment, or billing decisions. Ontology coverage and relational richness differ substantially across splits: CIM-10 contains semantic relations beyond hierarchy, while CCAM and ATC are predominantly hierarchical. The corpus is French and reflects the terminology versions used during generation.

Licensing

This dataset contains transformations of several independently licensed terminologies and therefore does not have a single permissive license. Use and redistribution remain subject to the applicable source terms:

• —CIM-10 FR PMSI (2025-01-01): CC BY-NC-ND 3.0 IGO, maintained by ATIH from the WHO classification.
• —CCAM v80.00: Licence Ouverte 2.0, maintained by CNAM.
• —ATC 2025-02: CC BY-ND 3.0 IGO, produced by WHO and translated by ANSM.

Users are responsible for determining whether their intended use and redistribution comply with these terms.

Citation

If you use this dataset, please cite the OntoBook paper:

@inproceedings{touchent:hal-05697506,
  TITLE = {{OntoBook: Ontology-Grounded Synthetic Textbooks for Medical Encoder Pretraining}},
  AUTHOR = {Touchent, Rian and de la Clergerie, {\'E}ric},
  URL = {https://hal.science/hal-05697506},
  BOOKTITLE = {{Proceedings of Knowledge Graphs and Large Language Models Workshop}},
  ADDRESS = {Palma de Mallorca, Spain},
  YEAR = {2026},
  MONTH = May,
  PDF = {https://hal.science/hal-05697506v1/file/main.pdf},
  HAL_ID = {hal-05697506},
  HAL_VERSION = {v1},
}

Paper