ClarusC64/clinical-reasoning-benchmark-v0.2

Clinical Reasoning Benchmark v0.2

This benchmark evaluates clinical reasoning quality under uncertainty.

The focus is not diagnosis.

The focus is whether a model can reason about stability, trajectory, recovery, boundary pressure, and intervention choice.

Included Datasets

What v0.2 Adds

v0.2 is harder than v0.1.

It adds:

• —counterfactual twin cases
• —misleading improvement cases
• —same-score different-label cases
• —visible severity versus trajectory conflicts
• —reserve capacity signals
• —support dependency signals
• —intervention comparison tasks

The benchmark is designed to reduce shortcut learning.

Capability Areas

The benchmark evaluates:

• —hidden support detection
• —silent deterioration detection
• —escalation discipline
• —recovery geometry
• —constraint pressure
• —margin awareness
• —recovery burden estimation
• —trajectory awareness
• —competing intervention selection

Intended Use

Use this benchmark to evaluate whether a model can reason about clinical stability rather than simply classify visible severity.

A model should not succeed by relying on one variable.

Good performance requires integrating:

• —current state
• —trend direction
• —treatment response
• —support dependence
• —reserve capacity
• —boundary proximity
• —recovery burden
• —competing intervention effects

Evaluation

Each dataset contains its own scorer.

Binary tasks report:

• —accuracy
• —precision
• —recall
• —f1
• —confusion matrix

Three-class tasks report:

• —accuracy
• —macro precision
• —macro recall
• —macro f1
• —confusion matrix

For full-suite reporting, group results by capability domain.

Structural Note

This benchmark evaluates reasoning fidelity through clinical scenarios.

The datasets expose observable variables and labels only.

They do not expose generator logic, hidden scoring rules, or latent construction methods.

License

MIT