hugo/healthbench-br
HealthBench-BR A true/false benchmark of 1,780 paired clinical assertions grounded in Brazil's official clinical guidelines (PCDTs), published by the Ministry of Health. For each clinical fact, two statements are provided: a true version faithful to the guideline, and a false version that modifies a single critical detail (a dosage, route of administration, monitoring interval, etc.). The dataset is perfectly balanced (50% true / 50% false), so correct classification requires… See the full description on the dataset page: https://huggingface.co/datasets/hugo/healthbench-br.
HealthBench-BR
A true/false benchmark of 1,780 paired clinical assertions grounded in Brazil's official clinical guidelines (PCDTs), published by the Ministry of Health.
For each clinical fact, two statements are provided: a true version faithful to the guideline, and a false version that modifies a single critical detail (a dosage, route of administration, monitoring interval, etc.). The dataset is perfectly balanced (50% true / 50% false), so correct classification requires precise factual recall — not surface heuristics. Evaluation extracts a Verdadeiro / Falso verdict from the model's response via regex.
- 1,780 items total
- 10 paired true/false statements per guideline · 178 guidelines total
- Train/test split by guideline (890/890 per split, 1780 total per split)
Usage
from datasets import load_dataset
ds = load_dataset("hugo/healthbench-br")Results — open models from this project
Test-split accuracy (greedy decoding). All checkpoints are publicly available:
Frontier models (for reference)
Citation
See the paper and code at the project repository:
Code & paper: https://github.com/hugoabonizio/clinical-protocols-br