Wilhelm-Foundation/rare-archive-synthetic-patients

Rare Archive Synthetic Patients — SFT Training Data

12,984 synthetic rare disease patient vignettes generated from Orphanet disease profiles. Designed for supervised fine-tuning (SFT) of diagnostic AI models. Part of the Rare AI Archive.

All patients are computationally generated. Zero real patient data. Zero PHI. This dataset contains no Protected Health Information. Every vignette is synthetically generated from public Orphanet disease profiles using frequency-weighted phenotype sampling. No real patients were involved in any stage of data creation.

Research use only. This dataset is training data for AI research. It is NOT intended for clinical decision-making.

Dataset Description

• —Repository: Wilhelm-Foundation/rare-archive-synthetic-patients
• —License: CC BY-NC-SA 4.0
• —Version: 0.1.0
• —Part of: Rare AI Archive · Complete Toolkit Collection

Generation Methodology

Pipeline Overview

Orphanet Disease Profiles (Orphadata API)
    ↓
HPO Phenotype Enrichment (rd-phenotypes endpoint)
    ↓
Frequency-Weighted Symptom Sampling
    ↓
Difficulty Tier Assignment (easy / medium / hard)
    ↓
Clinical Vignette Generation
    ↓
Family History Generation
    ↓
12,984 Synthetic Patient Records

Detailed Process

1. 1.Disease profile fetching: ~4,500 rare disease profiles retrieved from the Orphadata rd-phenotypes endpoint, each with associated HPO (Human Phenotype Ontology) term annotations and frequency data
2. 2.Frequency-weighted HPO sampling: For each synthetic patient, HPO terms are sampled based on their documented frequency in the disease profile (obligate > very frequent > frequent > occasional > very rare)
3. 3.Difficulty tiers: Each patient is assigned one of three difficulty levels:
4. 4.Easy: Core phenotypic features prominent, classic presentation
5. 5.Medium: Mix of core and variable features, some atypical elements
6. 6.Hard: Atypical presentation, overlapping phenotypes, diagnostic distractors
7. 7.Vignette generation: Sampled HPO terms composed into naturalistic clinical narrative with age, sex, and presentation context
8. 8.Family history: Generated to reflect inheritance patterns documented in the disease profile

Data Quality Notes

• —Vignettes are generated programmatically, not by LLM — ensuring reproducibility
• —Difficulty distribution is approximately balanced across tiers
• —Each disease may have 1-10 synthetic patients depending on phenotype richness

Ecosystem Context

Synthetic patients are the foundation of the training flywheel. Generated from Orphanet disease profiles across ~4,500 rare diseases, they provide the broad disease coverage that no single institution could collect on its own.

This dataset works in concert with the rest of the ecosystem:

• —Context Creators (clinicians, patient advocates) contribute structured vignettes from Undiagnosed Patient Hackathons that complement synthetic coverage with real diagnostic reasoning patterns
• —Validators evaluate model outputs in the ELO Arena, and their corrections are exported as additional training data that augments this dataset
• —Model Builders use this dataset plus Arena corrections to train condition-specific adapters for disease clusters like IEM, Neuromuscular, and more

The correction-to-retrain cycle means this dataset grows smarter with every clinician interaction — corrections are merged with synthetic cases for the next training run.

Dataset Structure

Data Fields

Data Splits

Intended Use

Primary use: Stage 1 SFT (Supervised Fine-Tuning) training data for rare disease diagnostic models.

• —Fine-tune language models to recognize rare disease presentations
• —Augment real clinical case data (RareArena RDS/RDC) with broader disease coverage
• —Pre-training exposure to rare disease phenotype patterns

NOT intended for: Clinical decision-making, patient diagnosis, or evaluation benchmarking (use RDS or RDC for evaluation).

Part of the ecosystem flywheel: Synthetic patients → model training → Arena evaluation → clinician corrections → merged back into training data → better models. Each cycle widens disease coverage and deepens diagnostic accuracy.

Bias, Risks & Limitations

Known Biases

Risks

• —Training on synthetic data only is insufficient — models trained exclusively on synthetic vignettes will learn simplified patterns. Real clinical data (RDS/RDC) is essential for evaluation and supplementary training.
• —Vignette fidelity: Synthetically generated presentations may not capture the diagnostic complexity of real patients
• —Ground truth quality: Disease labels are inherited from Orphanet; misclassifications in source data propagate

Limitations

• —Monolingual (English only)
• —No structured lab results (symptom-based vignettes only)
• —No temporal progression — each vignette is a snapshot
• —Difficulty tiers are heuristic, not clinically validated
• —Some diseases have very few synthetic patients (1-2) due to sparse phenotype profiles

Loading the Dataset

Using HuggingFace Datasets

from datasets import load_dataset

# Load the full dataset
dataset = load_dataset("Wilhelm-Foundation/rare-archive-synthetic-patients")

# Access the training split
train = dataset["train"]
print(f"Total patients: {len(train)}")

# Inspect a single patient
patient = train[0]
print(f"Diagnosis: {patient['ground_truth_diagnosis']}")
print(f"Difficulty: {patient['difficulty']}")
print(f"HPO terms present: {len(patient['hpo_terms_present'])}")
print(f"Vignette: {patient['clinical_vignette'][:200]}...")

Using Pandas

import pandas as pd

df = pd.read_parquet(
    "hf://datasets/Wilhelm-Foundation/rare-archive-synthetic-patients/data/train-00000-of-00001.parquet"
)
print(f"Shape: {df.shape}")
print(f"Unique diseases: {df['disease_id'].nunique()}")
print(f"Difficulty distribution:\n{df['difficulty'].value_counts()}")

Filtering by Difficulty

from datasets import load_dataset

dataset = load_dataset("Wilhelm-Foundation/rare-archive-synthetic-patients", split="train")

# Get only hard cases for challenging fine-tuning
hard_cases = dataset.filter(lambda x: x["difficulty"] == "hard")
print(f"Hard cases: {len(hard_cases)}")

Preparing for SFT

from datasets import load_dataset

dataset = load_dataset("Wilhelm-Foundation/rare-archive-synthetic-patients", split="train")

# Convert to chat format for SFT
def to_chat_format(example):
    return {
        "messages": [
            {"role": "system", "content": "You are a rare disease specialist. Given a clinical presentation, provide your differential diagnosis with reasoning."},
            {"role": "user", "content": example["clinical_vignette"]},
            {"role": "assistant", "content": f"Based on the clinical presentation, my primary diagnosis is {example['ground_truth_diagnosis']}."}
        ]
    }

sft_dataset = dataset.map(to_chat_format)

Related Resources

Citation

@misc{rare-archive-synthetic-patients,
  title={Rare Archive Synthetic Patients: Frequency-Weighted HPO Sampling for Rare Disease SFT},
  author={Wilhelm Foundation},
  year={2026},
  publisher={HuggingFace},
  url={https://huggingface.co/datasets/Wilhelm-Foundation/rare-archive-synthetic-patients}
}

A program of the [Wilhelm Foundation](https://wilhelm.foundation). Built on [Lattice Protocol](https://github.com/LatticeProtocol). No disease is too rare to matter.