saxon11/fraudlens-icd10-llama3.1-8b-lora

FraudLens: ICD-10 Clinical Coding Model

LoRA adapter for Llama 3.1 8B that predicts ICD-10-CM diagnosis codes from hospital discharge summaries. Built as the core model for a clinical-coding consistency auditor.

Quick Start

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = "unsloth/meta-llama-3.1-8b-instruct-unsloth-bnb-4bit"
adapter = "saxon11/fraudlens-icd10-llama3.1-8b-lora"

tokenizer = AutoTokenizer.from_pretrained(base_model)
model = AutoModelForCausalLM.from_pretrained(base_model, load_in_4bit=True, device_map="auto")
model = PeftModel.from_pretrained(model, adapter)

messages = [
    {"role": "system", "content": "You are a clinical coding auditor..."},
    {"role": "user", "content": "<your discharge summary>"},
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True)
outputs = model.generate(inputs.to(model.device), max_new_tokens=200, temperature=0.3)
print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))

Output Format

<reasoning>Patient presented with acute respiratory failure requiring BiPAP.
Acute kidney injury with creatinine 3.2. Type 2 diabetes managed with insulin.
Chronic hypertension on home meds.</reasoning>
<codes>
<code>J9601</code>
<code>N179</code>
<code>E119</code>
<code>I10</code>
</codes>

Evaluation (50-case held-out test set)

The model is conservative — strong precision but under-predicts acute conditions. It reliably captures chronic conditions (hypertension, diabetes, hyperlipidemia) but often misses acute organ failures and metabolic derangements.

Training

Two-phase approach:

1. 1.SFT Warm-Start (200 steps): Supervised fine-tuning on filtered high-acuity MIMIC-IV cases to teach ICD-10 vocabulary and output format
2. 2.GRPO Reinforcement Learning (750 steps): Group Relative Policy Optimization with multi-component reward:
3. 3.XML structure reward (0-0.125)
4. 4.Format compliance — reasoning + codes (0-0.5)
5. 5.Valid ICD-10 code reward (0-1.0)
6. 6.Two-level F1 — exact match + category-level (0-6.5)
7. 7.Brevity penalty (prevents code dumping)
8. 8.Repetition penalty (prevents duplicate codes)

Training data: MIMIC-IV discharge summaries (PhysioNet) with linked ICD-10-CM codes. ~92K training examples, 50 held-out test cases.

Hyperparameters

Training Journey

14 iterations of reward engineering across 4 weeks:

• —Runs 1-3: Pure GRPO failed (F1 = 0.026). Model couldn't learn ICD-10 codes from scratch.
• —Run 7: SFT warm-start breakthrough (test F1 = 0.303), but format collapsed.
• —Runs 8-13: Iterative reward fixes — soft format multiplier, brevity penalty, filtered SFT data.
• —Run 14-15: Best balanced results. Filtered SFT on high-acuity cases + anti-repetition + relaxed brevity.

Limitations

• —Under-prediction: Averages 3.4 codes vs 5.6 true. Misses acute conditions.
• —Single-center training data: MIMIC-IV is from Beth Israel Deaconess only.
• —Research only: Not validated for clinical use.
• —Generic reasoning: Tends to use template reasoning rather than case-specific analysis.

Framework Versions

• —Transformers: 4.57.1
• —PEFT: 0.18.1
• —TRL: 0.23.1
• —PyTorch: 2.9.0+cu128
• —Unsloth: latest