medical-benchmark
synthetic-medical-document-recognition-benchmark
Synthetic Medical Document Recognition Benchmark
This dataset contains synthetic, English-language medical records rendered as
documents for evaluating automated data extraction and de-identification
systems. Each synthetic patient has a longitudinal FHIR R4 record and multiple
visual representations derived from that record.
Every rendered document is clearly marked as synthetic. This makes the dataset
suitable for manual testing, product demonstrations, and workflows that… See the full description on the dataset page: https://huggingface.co/datasets/morzel85/synthetic-medical-document-recognition-benchmark.llm-medical-reasoning-steps-benchmark
LLM Medical Reasoning Steps Benchmark
This dataset contains 1,170 medical reasoning benchmark questions with final answers, reference reasoning steps, and reference key points.
Dataset Files
data/all.jsonl: all 1,170 examples.
data/mcq.jsonl: 592 multiple-choice examples.
data/oeq.jsonl: 578 open-ended examples.
No model prediction outputs are included in this release.
Schema
Each JSONL row has the following fields:
{
"id": "mcq_0001",
"task_type":… See the full description on the dataset page: https://huggingface.co/datasets/medreason/llm-medical-reasoning-steps-benchmark.medical-chronology-benchmark
Medical Chronology LLM Benchmark
Can LLMs reliably extract structured medical timelines from unstructured records?
This dataset provides the golden ground truth, synthetic source documents, and pre-generated model outputs for benchmarking LLMs on medical chronology extraction — a critical task in medical-legal case review.
📦 GitHub (full code + evaluation pipeline): superinsight/superinsight-ai-benchmark
Key Results
Tier
Models
Composite
F1
Hallucination… See the full description on the dataset page: https://huggingface.co/datasets/Superinsight/medical-chronology-benchmark.medical-text-tabular-benchmark-2024
Medical Text Tabular Data Notes
Dataset summary
This repository contains a preparation pipeline and a small metadata sample for Medical work with Text Tabular inputs. It does not claim to be a complete benchmark release; the loader documents how source data is normalized and validated.
Included material
clean.py — loading, cleaning, and split preparation code.
dataset_infos.json — schema and split metadata.
metadata_sample.jsonl — small… See the full description on the dataset page: https://huggingface.co/datasets/yusufhandayani/medical-text-tabular-benchmark-2024.Medical_Ethical_Dilemmas_Benchmark
ClinicalEthicsBench revision analysis package
This directory is a publication-ready staging copy of the data and code used
for the primary five-model analyses in the JMIR AI revision. The source files
elsewhere in the manuscript workspace were copied, not moved or edited.
Scope
Primary panel: GPT-5, Claude Sonnet 4.5, Gemini 2.5 Pro, DeepSeek-R1, and
Meta-Llama-3-8B-Instruct.
Design: 60 cases, 3 trials per model, temperature 0.
Primary outcome: trial-level binary… See the full description on the dataset page: https://huggingface.co/datasets/MedicalAILabo/Medical_Ethical_Dilemmas_Benchmark.medical-multimodal3-benchmark
Medical Multimodal3 Data Notes
Dataset summary
A documented Medical data-preparation workflow for Multimodal3 records. The bundled rows demonstrate the schema and validation path rather than pretending to be a full training corpus.
Included material
build_dataset.py — loading, cleaning, and split preparation code.
dataset_infos.json — schema and split metadata.
metadata_sample.jsonl — small, human-readable records for checking the schema.
README.md… See the full description on the dataset page: https://huggingface.co/datasets/anildezf/medical-multimodal3-benchmark.