Ailiance-fr/qwen3-4b-mascarade-stm32-lora

Model Card for qwen3-4b-mascarade-stm32-lora

This model is a fine-tuned version of Qwen/Qwen3-4B. It has been trained using TRL with SFT on a STM32 firmware corpus as part of the Ailiance mascarade LoRA family.

Quick start

from transformers import pipeline

question = "Write STM32 HAL code to configure UART2 at 115200 baud with DMA RX."
generator = pipeline("text-generation", model="Ailiance-fr/qwen3-4b-mascarade-stm32-lora", device="cuda")
output = generator([{"role": "user", "content": question}], max_new_tokens=256, return_full_text=False)[0]
print(output["generated_text"])

Bench results — iact-bench Phase 8 (Docker validators, 2026-05-11)

Functional eval via the `iact-bench` v0.2 Docker sandbox validators (no-network, read-only rootfs, uid 1000 dropped caps, --tmpfs /tmp:size=256m). Source LoRA: Ailiance-fr/qwen3-4b-mascarade-stm32-lora. Eval samples drawn with seed=101 from Ailiance-fr/mascarade-stm32-dataset.

Pass = sandboxed compile/exec succeeds (exitcode=0). See iact-bench/configs/domain_validators.yaml for the exact toolchain invocation (arm-none-eabi-gcc / arduino-cli / freecadcmd depending on domain)._

Upstream base model — official evaluations

These are the official scores for the unmodified base model `Qwen/Qwen3-4B-Instruct-2507`, reported by Alibaba Qwen team. They represent the floor of capability that this LoRA inherits before the hardware-domain fine-tune adapts behavior.

Source: official Qwen3-4B-Instruct-2507 model card.

Reading these numbers alongside the Phase 6 bench above: the upstream scores measure general capability (knowledge, reasoning, coding, alignment). The Phase 6 deltas measure hardware-domain specialization (KiCad, SPICE, schematic extraction). A rank-16 LoRA adapter modifies less than 1% of base weights, so the upstream scores remain approximately the floor — this LoRA adds the Phase 6 deltas on top of these inherited capabilities.

Training procedure

This model was trained with SFT on a STM32 firmware corpus.

Framework versions

• —TRL: 1.4.0
• —Transformers: 5.8.0
• —Pytorch: 2.11.0
• —Datasets: 4.8.5
• —Tokenizers: 0.22.2

Bench results — held-out token-overlap (evalmascaradelora, n=10)

Evaluated on 10 random held-out prompts from Ailiance-fr/mascarade-stm32-dataset (seed=101 ≠ train seed 42).

Token-overlap is a coarse quality proxy — high overlap (>0.4) suggests the LoRA reproduces domain vocabulary; low overlap indicates either domain-shift or stylistic divergence from the reference. See `ailiance/ailiance-bench` for richer functional evaluations (KiCad DRC, SPICE convergence, etc.) on the same family.

Citations

@software{vonwerra2020trl,
  title   = {{TRL: Transformers Reinforcement Learning}},
  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
  license = {Apache-2.0},
  url     = {https://github.com/huggingface/trl},
  year    = {2020}
}

Bench (vs base Qwen3-4B)

Consolidated comparison of this LoRA against its base, drawing on two complementary evaluation streams. The reference base used for cross-adapter comparison in Phase 6 is gemma-e4b-eu-kiki-base (legacy Gemma-4 ancestor). A dedicated Qwen3-4B-Instruct-2507 baseline run is not in our pipeline yet — those rows are n/a.

Phase 6 — cross-adapter scoreboard (reference base: gemma-e4b-eu-kiki-base)

Methodology: iact-bench v0.2.0 (audit-grade Docker validators), greedy decoding, maxtokens per `GENPARAMS. NDJSON audit trail in [ailiance/ailiance-bench](https://github.com/ailiance/ailiance-bench). **Scoring date:** 2026-05-11 (commit [46801af`](https://github.com/ailiance/ailiance-bench/commit/46801af)).

Phase 6 numbers reflect adapter behavior on a Gemma-4 reference base; domain semantics transfer to the Qwen3-4B production base served via Tower Ollama :8004, but absolute scores may shift. A Qwen3-4B baseline run is tracked for a future bench refresh.

Cross-domain forgetting check (Phase 9, 2026-05-11)

For each domain's eval set (seed=101, n samples held-out), compare this LoRA's Jaccard token-overlap vs the Qwen3-4B-Instruct-2507 baseline (no adapter) on the SAME prompts. Negative Δ = the LoRA degrades base behaviour on that domain.

In-domain Δ: 0.009 Out-of-domain mean Δ: 0.004