Ailiance-fr/devstral-cpp-lora

1---2license: apache-2.03base_model: mistralai/Devstral-Small-2-24B-Instruct-25124library_name: peft5tags:6- mlx7- lora8- peft9- ailiance10- devstral11- cpp12language:13- en14- fr15pipeline_tag: text-generation16---17 18# Ailiance — Devstral-Small-2-24B-Instruct cpp LoRA19 20LoRA adapter fine-tuned on `mistralai/Devstral-Small-2-24B-Instruct-2512` for **cpp** tasks.21 22> Maintained by **Ailiance** — French AI org publishing EU AI Act aligned LoRA adapters and datasets.23 24## Quick start (MLX)25 26```python27from mlx_lm import load, generate28 29model, tokenizer = load(30    "mistralai/Devstral-Small-2-24B-Instruct-2512",31    adapter_path="Ailiance-fr/devstral-cpp-lora",32)33 34print(generate(model, tokenizer, prompt="..."))35```36 37## Training38 39| Hyperparameter   | Value                  |40|------------------|------------------------|41| Base model       | `mistralai/Devstral-Small-2-24B-Instruct-2512`     |42| Method           | LoRA via `mlx-lm`      |43| Rank             | 16            |44| Scale            | 2.0           |45| Alpha            | 32           |46| Max seq length   | 2048  |47| Iterations       | 500           |48| Optimizer        | Adam, LR 1e-5          |49| Hardware         | Apple M3 Ultra 512 GB  |50 51## Training data lineage52 53Derived from the internal **eu-kiki / mascarade** curation. All upstream samples54are synthetic, permissively-licensed, or generated from Apache-2.0 base resources.55See the [Ailiance-fr catalog](https://huggingface.co/Ailiance-fr) for related cards.56 57## Training metrics58 59Extracted from training log (`batch_eu_kiki_v2.log`):60 61| Metric | Value |62|---|---:|63| Final train loss | 0.603 |64| Final validation loss | 0.401 |65| Val loss reduction | +1.779 (from 2.180) |66| Iterations completed | 500 |67| Trainable parameters | 0.224% (279.708M / 125025.989M) |68 69> Validation loss is measured every 200 iterations on a held-out split of the70> training corpus (`val_batches=5`, `mlx-lm` LoRA trainer).71 72## Benchmark on production tasks73 74This LoRA has **not yet been evaluated** through the75[`electron-bench`](https://github.com/ailiance/ailiance-bench/blob/main) functional benchmark76pipeline. The current pipeline targets the `gemma-4-E4B` base only; support for77the **devstral** base is on the roadmap78([open issues](https://github.com/ailiance/ailiance-bench/issues)).79 80For a comparable reference matrix on a related domain (electronics, embedded,81KiCad), see the Gemma champions:82 83| Adapter | Highlights |84|---|---|85| [`Ailiance-fr/gemma-4-E4B-eukiki-lora`](https://huggingface.co/Ailiance-fr/gemma-4-E4B-eukiki-lora) | +55 P1-DSL, +42 P1-PCB, +25 SPICE, +38 P3 |86| [`Ailiance-fr/gemma-4-E4B-mascarade-lora`](https://huggingface.co/Ailiance-fr/gemma-4-E4B-mascarade-lora) | +48 P3 extraction |87 88Full base-vs-LoRA matrix: [`compare_base_vs_lora.md`](https://github.com/ailiance/ailiance-bench/blob/main/bench-results/compare_base_vs_lora.md).89 90## License chain91 92| Component                         | License           |93|-----------------------------------|-------------------|94| Base model (`mistralai/Devstral-Small-2-24B-Instruct-2512`)        | apache-2.0    |95| Training data (internal Ailiance curation (synthetic + permissive sources))         | apache-2.0      |96| **LoRA adapter (this repo)**      | **apache-2.0**|97 98_All upstream components are Apache 2.0 / MIT — LoRA inherits permissive terms._99 100## EU AI Act compliance101 102- **Article 53(1)(c)**: training data licenses preserved (per-dataset cards declare upstream licenses).103- **Article 53(1)(d)**: training data summary — see upstream dataset cards on Ailiance-fr.104- **GPAI Code of Practice (July 2025)**: base `mistralai/Devstral-Small-2-24B-Instruct-2512` released under apache-2.0.105- **No web scraping by Ailiance**, **no licensed data**, **no PII**.106- Upstream Stack Exchange content (where applicable) is CC-BY-SA-4.0 and propagates to this adapter.107 108## License109 110LoRA weights: **apache-2.0** — see License chain table above for derivation rationale.111 112## Citation113 114```bibtex115@misc{ailiance_devstral_cpp_2026,116  author    = {Ailiance},117  title     = {Ailiance — Devstral-Small-2-24B-Instruct cpp LoRA},118  year      = {2026},119  publisher = {Hugging Face},120  url       = {https://huggingface.co/Ailiance-fr/devstral-cpp-lora}121}122```123 124## Related125 126See the full [Ailiance-fr LoRA collection](https://huggingface.co/Ailiance-fr).127 128 129## Bench comparison (2026-05-11)130 131### Base model (Devstral-Small-2-24B-MLX-4bit) capability132 133| Task | Score | Notes |134|---|---:|---|135| GSM8K-CoT flex EM | **0.96** | W3 lm-eval-harness (--limit 100) |136| ARC-Easy acc / acc_norm | **0.80 / 0.75** | |137| MMLU-Pro Computer Science | **0.64** | |138 139Source: <https://github.com/ailiance/ailiance/tree/main/output/lm-eval-base-2026-05-11>140 141### This LoRA (tuned) — bench PENDING142 143Will include kicad-sch / iact-bench validators + W3 lm-eval delta. See spec for144methodology:145<https://github.com/ailiance/ailiance-bench/blob/main/docs/superpowers/specs/2026-05-11-kicad-sch-gap-design.md>146 147## Upstream base model — official evaluations148 149This LoRA fine-tunes [`mistralai/Devstral-Small-2-24B-Instruct-2512`](https://huggingface.co/mistralai/Devstral-Small-2-24B-Instruct-2512),150Mistral's coding-specialist LLM. Headline software-engineering benchmarks151from the upstream model card:152 153| Benchmark                | Devstral Small 2 (24B) | Devstral 2 (123B) | DeepSeek v3.2 (671B) | Claude Sonnet 4.5 |154|--------------------------|-----------------------:|------------------:|---------------------:|------------------:|155| **SWE Bench Verified**   | **68.0 %**             | 72.2 %            | 73.1 %               | 77.2 %            |156| **SWE Bench Multilingual** | **55.7 %**           | 61.3 %            | 70.2 %               | 68.0 %            |157| **Terminal Bench 2**     | **22.5 %**             | 32.6 %            | 46.4 %               | 42.8 %            |158 159(For reference, GPT-5.1 Codex High: 73.7 % SWE Verified · 52.8 % Terminal Bench 2.)160 161Devstral Small 2 (24B) is competitive with much larger open models on162SWE Bench Verified (e.g. matches GLM-4.6 at 355B). Architecture uses163rope-scaling per Llama 4 + Scalable-Softmax ([arXiv:2501.19399](https://arxiv.org/abs/2501.19399)).164 165**Source:** [official Devstral-Small-2-24B-Instruct-2512 model card](https://huggingface.co/mistralai/Devstral-Small-2-24B-Instruct-2512).166 167> **Reading these alongside this LoRA:** Devstral Small 2 is a strong168> coding base. This LoRA inherits its SWE-Bench performance and adds169> language- or domain-specific specialization.170