tugot17/lfm2-8b-a1b-smoltalk-lora-classic

LFM2-8B-A1B SmolTalk LoRA — classic per-expert format

Per-expert (Mixtral-style) LoRA adapter for `LiquidAI/LFM2-8B-A1B`, produced by converting the `LiquidAI/LFM2-8B-A1B-smoltalk-LoRA` adapter (trained via PEFT 0.18+'s target_parameters on the batched 3D MoE expert tensors) into the classic per-expert layout (...mlp.experts.<N>.gate_proj/up_proj/down_proj.lora_*).

The conversion is mathematically lossless — PEFT's fused-parameter LoRA over experts.gate_up_proj decomposes into per-expert (A, Bgate || Bup) blocks; only the expert tensors are repacked, every other weight is carried over verbatim. See `convert_lfm2_moe_lora_to_per_expert.py`.

What's in this adapter (full coverage of the original)

• —Attention (all attention layers): q_proj, k_proj, v_proj, out_proj
• —ShortConv (all conv layers): in_proj, out_proj
• —Dense MLP (layers 0–1): gate_proj, up_proj, down_proj (renamed from LFM2-native w1/w3/w2)
• —MoE router (layers 2–23): gate
• —MoE experts (layers 2–23, 32 experts each): per-expert gate_proj, up_proj, down_proj

r=8, alpha=16, num_experts=32, hidden=2048, moe_intermediate=1792. Per-expert gate and up share lora_A (PEFT's fused training has a single shared A; splitting B's output axis into gate || up halves and keeping A common is the per-expert equivalent).

Usage with SGLang

Requires SGLang with the LFM2 LoRA wiring patch (tugot17/sglang#10). Once it lands upstream, stock SGLang works.

sglang serve \
    --model-path LiquidAI/LFM2-8B-A1B \
    --port 30000 \
    --enable-lora --max-lora-rank 8 \
    --lora-paths "smoltalk=tugot17/lfm2-8b-a1b-smoltalk-lora-classic" \
    --lora-target-modules q_proj k_proj v_proj out_proj gate_proj up_proj down_proj in_proj gate \
    --lora-strict-loading

Generate with LoRA:

curl -sS http://localhost:30000/generate \
    -H "Content-Type: application/json" \
    -d '{
      "text": "Write a short story set in an unusual location.",
      "sampling_params": {"temperature": 0.0, "max_new_tokens": 80},
      "lora_path": "smoltalk"
    }'

Correctness

Verified against HF+PEFT running the original, unfiltered adapter, by teacher-forced top-k logprob comparison (greedy, bf16): SGLang's 32-token generation is token-identical to HF and the logprob divergence (prefill mean 0.22 / max 0.77; decode mean 0.28 / max 0.87) is well below the base model's no-LoRA kernel-noise floor (prefill 0.37/1.50; decode 0.40/1.75). The expert-tensor repacking was additionally checked against PEFT's own merge_and_unload deltas — they match scale·(B@A) per expert to bf16 rounding (max abs err 5e-4).

Reproducing the conversion

huggingface-cli download tugot17/lfm2-8b-a1b-smoltalk-lora-classic convert_lfm2_moe_lora_to_per_expert.py --local-dir .
huggingface-cli download LiquidAI/LFM2-8B-A1B-smoltalk-LoRA --local-dir ./src
python convert_lfm2_moe_lora_to_per_expert.py --src ./src --dst ./out

Only depends on torch and safetensors.

Training (original adapter)

Per the parent adapter card: r=8, alpha=16, dropout=0, 3 epochs on a 5000-sample slice of HuggingFaceTB/smoltalk. All numbers carry over — the conversion doesn't change the LoRA semantics.