tugot17/lfm2.5-1.2b-instruct-smoltalk-lora-classic

LFM2.5-1.2B-Instruct SmolTalk LoRA — SGLang-ready

LoRA adapter for `LiquidAI/LFM2.5-1.2B-Instruct`, derived from `LiquidAI/LFM2.5-1.2B-Instruct-smoltalk-LoRA` by dropping the ShortConv (conv.in_proj / conv.out_proj) LoRA weights, which SGLang does not have an injection point for yet. All other weights are unchanged (native w1/w3/w2 MLP naming is kept — both PEFT and SGLang handle it).

See `convert_lfm2_moe_lora_to_per_expert.py` (bundled; the same script converts LFM2-MoE adapters to the per-expert format).

What's in this adapter

• —Attention (every attention layer): q_proj, k_proj, v_proj, out_proj
• —MLP (every layer): w1, w3, w2 (gate / up / down)

Dropped relative to the original (out of v1 scope): ShortConv conv.in_proj, conv.out_proj.

Usage with SGLang

Requires SGLang with the LFM2 LoRA wiring patch (tugot17/sglang#10). Once it lands upstream, stock SGLang works.

sglang serve \
    --model-path LiquidAI/LFM2.5-1.2B-Instruct \
    --port 30000 \
    --enable-lora --max-lora-rank 8 \
    --lora-paths "smoltalk=tugot17/lfm2.5-1.2b-instruct-smoltalk-lora-classic" \
    --lora-target-modules q_proj k_proj v_proj out_proj gate_proj up_proj down_proj \
    --lora-strict-loading

Generate with LoRA:

curl -sS http://localhost:30000/generate \
    -H "Content-Type: application/json" \
    -d '{
      "text": "Write a short story set in an unusual location.",
      "sampling_params": {"temperature": 0.0, "max_new_tokens": 80},
      "lora_path": "smoltalk"
    }'

Correctness

Verified against HF+PEFT by teacher-forced top-k logprob comparison (greedy, bf16), with the HF reference using the original adapter filtered to the same module set: SGLang's generation is token-identical to HF, and the logprob divergence matches the base model's no-LoRA noise floor.

Training (original adapter)

Per the parent adapter card: r=8, alpha=16, dropout=0.1, lr 5e-5, 1 epoch on a 5k-example slice of HuggingFaceTB/smoltalk. This artifact only removes the ShortConv weights; everything else carries over.