Datasetpublic

Giordanopsouza/clinicalbr

ClinicalBr ClinicalBr is the first bilingual (Portuguese–English) clinical-decision benchmark built from 2,892 real Brazilian case reports drawn from 36 open-access medical journals spanning 18 specialties. Every case is provided as a parallel PT/EN pair and supports four evaluation tasks. Please refer to the paper for full details on the tasks, methodology, and limitations. Tasks & metrics Task Config n / lang Metric Diagnosis retrieval diagnosis 2,135… See the full description on the dataset page: https://huggingface.co/datasets/Giordanopsouza/clinicalbr.

sourceHugging Facecc-by-nc-4.0updated 29d agoView on Hugging Face

0likes156downloads

Dataset Card

ClinicalBr

ClinicalBr is the first bilingual (Portuguese–English) clinical-decision benchmark built from 2,892 real Brazilian case reports drawn from 36 open-access medical journals spanning 18 specialties. Every case is provided as a parallel PT/EN pair and supports four evaluation tasks.

Please refer to the paper for full details on the tasks, methodology, and limitations.

Tasks & metrics

Task	Config	n / lang	Metric
Diagnosis retrieval	`diagnosis`	2,135	LLM-judge accuracy (correct/incorrect)
Differential diagnosis	`diagnosis` (gold differential + UMLS CUIs)	2,135	UMLS-CUI precision/recall/F1
Exam recommendation	`exams`	851	UMLS-CUI macro-F1 (842/844 have gold)
Treatment planning	`treatment`	2,041	LLM-judge 1–5 (accuracy / completeness / clarity)

The full un-masked parallel corpus is in the corpus config (2,892/lang).

Configs & splits

Each config has two splits, pt and en, with identical PID sets (parallel).

Config	Rows / lang	Description
`corpus`	2,892	Full structured case (presentation, workup, diagnosis, treatment)
`diagnosis`	2,135	Masked input + gold final diagnosis + gold differential + gold CUIs
`exams`	851	Masked input (presentation only) + gold exams by category + pooled CUIs
`treatment`	2,041	Masked input + gold treatment plan
`results_diagnosis`	2,135 × 4	Reference predictions + 3-judge verdicts (nested) + consensus
`results_exams`	851 × 4	Reference exam predictions + linked UMLS CUIs
`results_treatment`	2,041 × 4	Reference predictions + 4-judge scores (nested)

Citation

bibtex

@article{clinicalbr,
  title   = {Cross-Lingual Evaluation of Large Language Models on Real Brazilian Clinical Case Reports},
  author  = {Souza, Giordano and collaborators},
  year    = {2026},
  journal = {}
}