kpshinnik/repro-fhir-rl-tool-calling
Reproduction bundle — Reinforcement Learning for Tool-Calling Agents in FHIR Independent reproduction of ICML 2026 paper #33573 — Reinforcement Learning for Tool-Calling Agents in Fast Healthcare Interoperability Resources (FHIR) (Knorr, Müller, Bremer, Schweingruber), arXiv:2605.14126, OpenReview Ep6EcExLIY. The paper GRPO-post-trains a Qwen3-8B multi-turn CodeAct agent on FHIR-AgentBench (Lee et al. 2025, github), lifting answer correctness from 50 % (o4-mini) to 77 %. All… See the full description on the dataset page: https://huggingface.co/datasets/kpshinnik/repro-fhir-rl-tool-calling.
Reproduction bundle — Reinforcement Learning for Tool-Calling Agents in FHIR
Independent reproduction of ICML 2026 paper #33573 — Reinforcement Learning for Tool-Calling Agents in Fast Healthcare Interoperability Resources (FHIR) (Knorr, Müller, Bremer, Schweingruber), arXiv:2605.14126, OpenReview Ep6EcExLIY.
The paper GRPO-post-trains a Qwen3-8B multi-turn CodeAct agent on FHIR-AgentBench (Lee et al. 2025, github), lifting answer correctness from 50 % (o4-mini) to 77 %. All four challenge claims are marked abstract-only, and the authors released no code.
Reproducibility scope (honest)
Full reproduction of the 77 % headline is infeasible here: it needs GPU RL training (HF Jobs returns HTTP 402 – no credits), the SkyRL harness, and a 72B judge. HF Inference Providers is also 402. So we substitute the Gemini API (permitted by the challenge for LLM judges / agent-mechanism) and reproduce everything except the training loop, end-to-end on the real benchmark data:
- Benchmark grounding (Claims 1, 4) — the open MIMIC-IV-FHIR demo (100 patients) is FHIR-AgentBench's data: all 91/91 validation patients match. The val-set resource-type breakdown reproduces the paper's Fig. 6 exactly: Observation 155, Medication 71, Encounter 64 (Empty 131 vs paper 110 accounts for the 445-vs-424 val size).
- CodeAct agent mechanism (Claim 2) — a faithful re-implementation of the paper's multi-turn agent (tools
fhir_query/python/finish, ReAct loop, Qwen3 XML tool-call format, shared workspace) runs end-to-end over a local FHIR store, including the hardest MedicationRequest→Medication reference resolution. - Execution-grounded LLM-judge reward (Claim 3) — the FHIR-AgentBench judge prompt + binary reward
r∈{0,1}, shown to be semantic (accepts verbose/unit-different answers, rejects wrong/empty) — not static reference matching. Plus the GRPO advantage (eq 2) and DAPO zero-variance filter (math unit-tested). - Zero-shot baseline (partial Claims 1, 4) — a real small-scale eval of the substituted agent over stratified val questions gives an actual accuracy number (a baseline data point; NOT the RL-trained 77 %).
toy/substituted where noted; the RL-trained 77 % is not reproduced (documented blocker).
Contents
Reproduce
pip install pandas numpy mpmath matplotlib
# 1. FHIR-AgentBench data + MIMIC-IV-FHIR demo (open):
git clone https://github.com/glee4810/FHIR-AgentBench benchmark
curl -sL https://physionet.org/content/mimic-iv-fhir-demo/get-zip/2.1.0/ -o mimic.zip && unzip mimic.zip -d mimic_fhir
# 2. mechanism checks (no API):
python scripts/grpo.py
# 3. end-to-end eval (needs an LLM backend; we used Gemini keys):
python scripts/run_eval.py --per-cat 12 --max-turns 12