Apppublic

dmartinelli/growth-medqa-explorer

sourceHugging Facecc-by-4.0updated 26d agoView on Hugging Face

0likes

App README

Growth-MedQA Explorer (dashboard)

An interactive dashboard for filtering the 1,097-item Growth-MedQA benchmark by its two-axis taxonomy, derived USMLE physician-task analogues, verified HPO/MONDO identifiers, and HealthBench-inspired theme analogue.

This is the artifact behind the core paper claim: aggregate medical-QA accuracy masks clinically important weaknesses on pediatric growth-specific reasoning. Filter to z-score (A2) or anthropometry (A3) items and every model drops below its overall accuracy.

Run locally

bash

cd /Users/dominiero/Documents/GitHub/benchmark
python3 -m venv dashboard/.venv
source dashboard/.venv/bin/activate
python -m pip install -r dashboard/requirements.txt
streamlit run dashboard/app.py

Opens at http://localhost:8501.

What it does

Six tag layers drive the views:

—Topic hierarchy — L1 domain (A/B/C/D) → L2 subtopic (12 codes)
—Task schema — seven clinically interpretable task codes (T1–T7), from measurement/calculation through monitoring/referral
—Growth role — primary / clue / negative_control / distractor / absent
—HealthBench analogue — best-fit theme, fixed-choice applicability status, rationale, confidence, and prompt version
—USMLE analogue — transparent T-code-derived physician-task crosswalk, with T7 review-gated
—Verified identifiers — HPO phenotypes and MONDO diseases with exact OMIM/ORPHA cross-references

Tabs:

Tab	Purpose
Leaderboard	Filtered accuracy vs overall accuracy + Δ, per model, with Wilson CIs
Slice heatmap	Models × L2 topic, cell = accuracy or Δ-vs-overall
Error explorer	Items a strong model gets wrong while the majority get them right (the paper's headline error class), plus all-models-wrong
Item browser	Paginated wrapped item table with selectable columns, plus a detailed inspector with verified identifier provenance
Contribute	Clinician submission templates (MCQ item, longitudinal vignette) feeding a review queue; nothing writes to the canonical dataset live

The manuscript's external-comparison table (Growth-MedQA vs MedQA / MedMCQA / PubMedQA, this study's own 200-item non-pediatric samples) is not currently mirrored in the app.

The collapsed Advanced display and identifiers panel provides USMLE, HPO, and MONDO filters and controls which item-browser columns are visible. Identifier columns are hidden by default. Only source-grounded exact keyed-answer mappings and human-reviewed annotations appear in the app; pending ontology candidates are never loaded into the dashboard.

Files

dashboard/
  app.py                  Streamlit application
  taxonomy.py             L1/L2/T-code/growth-role label maps (mirror the scope doc)
  requirements.txt
  data/
    growth_medqa_items.parquet    copied from hf_upload/data/
    growth_medqa_scores.parquet   copied from hf_upload/data/
    growth_medqa_standard_annotations.parquet  verified annotation evidence/provenance

Refreshing the data

The dashboard data files and Hugging Face upload files are generated together with:

bash

/usr/local/bin/python3 scripts/prepare_hf_dataset.py

When a new model is evaluated, re-run the benchmark and prepare_hf_dataset.py; the dashboard picks up new correct__<Model> / pred__<Model> columns automatically.

Deploy (optional)

Streamlit Community Cloud can point at dashboard/app.py. Keep the Hugging Face dataset private unless the source-license review supports public release. Do not assume that a public dashboard may redistribute every source question merely because the taxonomy card is CC BY 4.0.

The item set is frozen, so no scheduled job is needed. Re-run scoring + prepare_hf_dataset.py only when adding a model, then push.

Status

Verified locally: data logic matches the manuscript analyses, the Streamlit AppTest harness runs all tabs and advanced filters without error, and the item browser wraps long text using a native paginated table. Deployed as a Hugging Face Space at https://huggingface.co/spaces/dmartinelli/growth-medqa-explorer (see the Deploy section for how to push updates).