HFXM/LLM4Variants-Llama-3.2-1B-Instruct

LLM4Variants-Llama-3.2-1B-Instruct

Dual-head sentence classifier for ACMG evidence-code + strength prediction on ClinVar submission comments. This is rank #2 of the grid search (ranked by joint test accuracy).

The model wraps the backbone `meta-llama/Llama-3.2-1B-Instruct` with two heads on top of mean-pooled hidden states:

• —code head — 28-way ACMG evidence code (PVS1, PS1–PS4, PM1–PM6, PP1–PP5, BA1, BS1–BS4, BP1–BP7 + NO_KEYWORD)
• —strength head — 6-way strength, conditioned on a learned embedding of the predicted code (Supporting, Moderate, Strong, VeryStrong, NotMet, NoStrength)

Test metrics

Training configuration

Files

• —model.safetensors — full state dict (backbone + code_head + code_embeddings + strength_head).
• —label_mappings.json — keyword2id / strength2id (and reverse).
• —tokenizer files + chat_template.jinja.

Loading

This is a custom nn.Module (DualHeadLLM), not a transformers AutoModel. Reconstruct the module (see train_dual_head.py), then load the weights:

from safetensors.torch import load_file
from huggingface_hub import hf_hub_download

model = DualHeadLLM("meta-llama/Llama-3.2-1B-Instruct", num_keywords=28, num_strengths=6)
state = load_file(hf_hub_download("HFXM/LLM4Variants-Llama-3.2-1B-Instruct", "model.safetensors"))
model.load_state_dict(state)