boods/EnToFrMedicaLLM-Multilingual
EnMed-Unified — French Medical LLM (Multi-Task)
Headline system of the EnMed family. A Qwen3-14B decoder adapted for French medical question answering through domain-adaptive continual pre-training (DAPT) on a large French health corpus, followed by multi-task LoRA fine-tuning across three QA formats simultaneously. Phase 1 evaluation establishes 4 statistically significant wins over the un-adapted Qwen3-14B-vanilla baseline (BH-corrected, q = 0.05) with zero significant losses across nine independent (task × shot) evaluation cells.
Model Family Overview
The EnMed family consists of five variants, all built on Qwen3-14B:
Intended Uses
Supported tasks
- French Medical Multiple-Choice QA — select the best answer from 4–5 candidates (e.g., medical licensing exam questions from FrenchMedMCQA / DrBenchmark)
- French Clinical Extractive QA — identify and return verbatim answer spans from French clinical case narratives (CAS corpus format)
- French Medical Abstractive QA — generate free-form answers to open-ended French medical questions (MediQAl format)
Out-of-scope uses
- ⚠️ Clinical decision support / patient-facing deployment — this is a research prototype. It has not been validated for real clinical use. Do not use outputs to guide patient care.
- English-only medical QA — the DAPT stage targets French; English capability may have drifted from the base model.
- Languages other than French — not evaluated.
- NER, summarisation, or classification — not part of the training or evaluation protocol.
Quick Start
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
model_id = "brice-eloundou/EnMed-Unified" # replace with your actual HF repo
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)
# ── Multiple-Choice QA ───────────────────────────────────────────────────────
prompt = """Tu es un expert médical francophone. Réponds à la question suivante
en choisissant la meilleure réponse parmi les options proposées.
Question: Quelle est la principale cause d'insuffisance rénale aiguë en réanimation ?
A) Glomérulonéphrite aiguë
B) Nécrose tubulaire aiguë ischémique
C) Pyélonéphrite aiguë
D) Lithiase urinaire
Réponse:"""
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
out = model.generate(**inputs, max_new_tokens=16, temperature=0.1, do_sample=False)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))Log-probability decoding (recommended for MCQA)
For evaluation and benchmarking, score each option under teacher forcing and select the highest-likelihood token — this matches the evaluation protocol used in the paper and avoids format-compliance failures.
import torch, torch.nn.functional as F
def score_option(model, tokenizer, prefix, option_text):
text = prefix + option_text
enc = tokenizer(text, return_tensors="pt").to(model.device)
prefix_len = tokenizer(prefix, return_tensors="pt")["input_ids"].shape[1]
with torch.no_grad():
logits = model(**enc).logits[0, prefix_len-1:-1]
option_ids = enc["input_ids"][0, prefix_len:]
lp = F.log_softmax(logits, dim=-1)
return lp[range(len(option_ids)), option_ids].sum().item()
options = {"A": "Glomérulonéphrite aiguë",
"B": "Nécrose tubulaire aiguë ischémique",
"C": "Pyélonéphrite aiguë",
"D": "Lithiase urinaire"}
scores = {k: score_option(model, tokenizer, prefix=prompt, option_text=v)
for k, v in options.items()}
print("Predicted:", max(scores, key=scores.get))Training Details
Base model
Qwen/Qwen3-14B — instruction-tuned release.
Stage 1 — Domain-Adaptive Continual Pre-training (DAPT)
The backbone undergoes continual pre-training on the French health corpus introduced by Mannion et al. (2026), a large openly licensed collection of French clinical and biomedical text. This stage uses no task supervision; it exposes the model to French medical vocabulary and discourse without committing to a downstream task format.
Stage 2 — Multi-Task LoRA Fine-tuning
A single LoRA adapter is trained jointly on all three downstream QA tasks, with task identifiers embedded in the prompt. This design prevents the length/style register over-fitting that degrades single-task adapters under LLM-as-judge evaluation (see Limitations).
Evaluation
All eight systems were evaluated on three French medical QA tasks under 0-shot, 3-shot, and 5-shot prompting — a 3 × 3 grid of nine independent (task, shot) cells. Item-level paired t-tests were conducted per cell against Qwen3-14B-vanilla, with Benjamini–Hochberg FDR control (q = 0.05) and Bonferroni bound reported alongside.
Raw scores across all models and shot counts
The dotted line marks the Qwen3-14B-vanilla 0-shot reference. EnMed variants consistently sit above or on the reference for MCQA and ExtQA; the AbsQA panel reveals the EnMed-AbsQA collapse discussed in Limitations.
Per-task means (averaged over 0 / 3 / 5-shot)
Global descriptive ranking (normalised, 9 cells)
This ranking is descriptive only — normalisation across incomparable metric scales does not constitute a significance test.
Normalised scores across all 9 (task × shot) cells
Per-cell deltas versus Qwen3-14B-vanilla
Item-level paired t-tests with 95 % confidence intervals
Positive bars mean the EnMed variant outperforms the reference; negative bars mean the opposite. Only starred bars represent statistically significant differences.
Significance heatmap — per-cell annotated deltas
Statistical significance record vs. Qwen3-14B-vanilla
(9 independent item-level paired t-tests; α = 0.05; BH-corrected wins marked)
Best model at every (task × shot) cell
No single system wins all nine cells: EnMed-AbsQA leads MCQA, EnMed-ExtQA leads 0- and 5-shot ExtQA, and AbsQA cells split across EnMed-DAPT, Qwen3-14B-vanilla and EnMed-MCQA. EnMed-Unified does not lead any single cell but is never the worst.
Critical Difference diagrams — rank analysis per shot count
Average rank across the three tasks (lower = better). Critical difference CD = 6.06.
The CD (6.06) exceeds the observed rank spread, so these diagrams are descriptive consensus rankings — they corroborate but do not independently prove the item-level findings above.
Limitations
Multiplicity. Benjamini–Hochberg correction at q = 0.05 confirms EnMed-Unified's four headline wins. Weaker cells (e.g., ExtQA-3, MCQA-5) do not survive correction and should be treated as suggestive.
Distributional assumptions. Paired t-tests assume approximately normal per-item differences, which may not hold for binary MCQA outcomes or ordinal 1–5 judge scores. A fully ordinal-aware treatment remains future work.
Single-judge evaluation. AbsQA scores were generated by a single Gemma-family LLM-as-judge. Single-judge evaluations are susceptible to judge-specific biases; a predominantly English-trained judge may under-reward answers correct under French clinical conventions. Judge diversity and order-invariance checks have not been conducted.
Task-specific adapter paradox. EnMed-AbsQA and EnMed-ExtQA improve MCQA while significantly degrading their own nominal home task under LLM-as-judge scoring. We attribute this to over-fitting to a length/style register the judge penalises. Multi-task training (EnMed-Unified) mitigates this.
Phase 2 not yet released. This is the Phase 1 model. The full cross-lingual continual pre-training pipeline (English biomedical → French medical transfer) will be released as EnMed-Phase2.
⚠️ Not for clinical deployment. This model has not been clinically validated. Do not use it for patient-facing applications or clinical decision support.
Citation
The associated paper has been submitted to Springer Lecture Notes in Computer Science (LNCS) and is currently under review. If you use EnMed-Unified or any member of the EnMed family, please cite the preprint version:
@unpublished{abodoeloundou2025enmed,
title = {Cross-Lingual Domain Adaptation and Multi-Task Fine-Tuning
for High-Fidelity Medical Language Models},
author = {Abodo Eloundou, Brice Donald and Malykh, Valentin},
note = {Submitted to Springer Lecture Notes in Computer Science (LNCS).
Under review. ITMO University / MTS Web Services,
Saint Petersburg, Russia},
year = {2026}
}This entry will be updated to a full `@inproceedings` citation upon acceptance.
If you use the French health pre-training corpus, please also cite:
@article{mannion2026biomedical,
title = {Is biomedical specialization still worth it?
Insights from domain-adaptive language modelling
with a new French health corpus},
author = {Mannion, A. and Macaire, C. and Violle, A. and
Ohayon, S. and Tannier, X. and Schwab, D. and others},
journal = {arXiv preprint arXiv:2604.06903},
year = {2026}
}Acknowledgements
Research conducted at ITMO University, Saint Petersburg, Russia and MTS Web Services, Saint Petersburg, Russia.
Authors:
- Brice Donald Abodo Eloundou — ITMO University | ORCID: 0009-0009-1845-5867
- Valentin Malykh — MTS Web Services / ITMO University
Evaluation benchmarks: DrBenchmark (Labrak et al., 2024), FrenchMedMCQA (Labrak et al., 2022), MediQAl (Bazoge, 2025), CAS corpus (Grabar et al., 2020).
License
Released under Apache 2.0, consistent with the Qwen3-14B base model license. The pre-training corpus license follows Mannion et al. (2026); users are responsible for compliance with that corpus's terms.
Clinical use warning: This model is a research artefact. Any use in clinical or patient-facing settings requires independent clinical validation and regulatory approval in the applicable jurisdiction.