Modelpublic

jmfraga/gpt-oss-20b-onco-lora

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

0likes

Model Card

gpt-oss-20b-onco-lora — Oncology QLoRA adapter (Spanish)

QLoRA adapter fine-tuning InferenceIllusionist/gpt-oss-20b-MLX-4bit on a Spanish synthetic oncology question-answer dataset derived from clinical guidelines (NCCN, ESMO, IMSS, national consensus). Architecture: Mixture-of-Experts, 20B.

WARNING: Released as a documented negative result and for reproducibility - NOT a production model. In our blinded study, domain fine-tuning reduced clinical answer quality vs. the un-augmented base model. Under an LLM judge (Claude Opus 4), this adapter scored 0.14/5 on a 4-criterion clinical rubric, vs —/5 for the same base model with no adapter. Do not use for clinical decision-making.

Intended use

—Reproducing the paper's experiments; studying why domain fine-tuning can degrade a strong base model.
—Not for clinical, diagnostic, or treatment decisions.

Training

—MLX (mlx-lm), QLoRA on a Mac Mini M4 (64 GB), 4-bit base.
—LoRA rank 64, all layers, prompt masking, dropout 0.05, scale 32.
—Learning rate 2e-5, 3000 iterations, max seq length 2048.
—Data: Spanish teacher-generated Q&A grounded in copyrighted guideline text (corpus itself not redistributable).

Usage (MLX)

bash

pip install mlx-lm
python -m mlx_lm.generate --model InferenceIllusionist/gpt-oss-20b-MLX-4bit --adapter-path ./ \
  --prompt "Paciente con adenocarcinoma pulmonar EGFR+ estadio IV. Primera linea?"

Evaluation

Setting	Composite (Opus judge, /5)
This adapter (fine-tuned)	0.14
Same base model, no adapter	—

Citation

Fraga Sastrias, J. M. et al. Base Language Models Outperform Retrieval-Augmentation and Fine-Tuning for Clinical Oncology. (2026). Code: https://github.com/jmfraga/medexpert-oncologia

License

Inherits the base model license (apache-2.0); adapter provided for research/reproducibility.