Rsan0948/medtech-acmg-8b-mlx

medtech-8b

LoRA adapter for mlx-community/Qwen3-8B-bf16, distilled from DeepSeek-R1 reasoning traces for 5-class germline variant classification under ACMG/AMP guidelines.

The model reads a structured variant trace (gene, consequence, allele frequency, ClinVar review status) and returns a JSON object with classification, triggered_criteria, reasoning_trace, and confidence.

Results

Comparators on the same validation split: rule-based baseline 75.51%, DeepSeek-R1 teacher 69.39%.

Pathogenic recall is 23/23 on validation and 7/7 on holdout. Full per-class precision/recall/F1, confusion matrices, and per-confidence breakdowns are in evaluation/summary.json of the accompanying release.

The holdout set is the honest number: the validation split was also used for checkpoint selection (best validation loss), so validation accuracy is mildly optimistic.

Usage

from mlx_lm import load, generate

model, tok = load(
    "mlx-community/Qwen3-8B-bf16",
    adapter_path="path/to/medtech-8b",  # this repo
)

Prompts are built with the PromptFactory in the source repository (src/modeling/prompt_factory.py), which formats a variant trace against docs/ACMG_GUIDELINES_V1.txt.

Label-only inference (edge deployment): the JSON schema emits "classification" first, so generation can be truncated after the first value. Measured time-to-label on an M1 Max: 3.97s (vs 30.25s for the full reasoning trace).

Training

• —Method: LoRA, rank 64, alpha 128, dropout 0.05, applied to 32 layers (~2.9% of parameters trainable)
• —Data: 715 distilled examples (DeepSeek-R1 teacher, agreement-filtered against ClinVar, prompts rebuilt hint-free after a label-leakage fix) + 98-example validation split
• —Schedule: cosine 2e-4 -> 1e-6, warmup 100, effective batch 16, max seq 2048, seed 42
• —Early-stopped at ~iter 360; shipped checkpoint = iter 200 (best val loss 0.382), md5-verified
• —Hardware: Apple M1 Max 64 GB, mlx-lm 0.31.3, ~8h50m wall clock

Full details in training_manifest.json and training_config.yaml.

Scope and limitations

• —Narrow domain: ~99% BRCA1/BRCA2 (plus 10 MLH1 variants). Not evaluated on other genes.
• —Research artifact. Not a medical device, not for clinical use.
• —Benign and Likely Pathogenic classes have very small test supports (2 and 1 on validation); treat those per-class metrics as unstable.
• —Reasoning traces agree with the teacher's triggered ACMG criteria at 0.78 mean Jaccard (validation). A 50-case blind community review of reasoning integrity is in progress (evaluation/reasoning-review.csv).

License

Apache-2.0 (adapter). Base model Qwen3 is Apache-2.0. Training data derives from NCBI ClinVar (freely available US government data), gnomAD (public), and DeepSeek-R1 teacher outputs (MIT).