Modelpublic

sahilchachra/Leanstral-1.5-119B-A6B-BF16

sourceHugging Faceotherupdated 2mo agoView on Hugging Face

3likes62downloads

Model Card

Leanstral-1.5-119B-A6B — BF16 (HF `transformers` format)

An unofficial, community conversion of `mistralai/Leanstral-1.5-119B-A6B` from Mistral's native (consolidated FP8) format into standard Hugging Face transformers format, dequantized to BF16. The upstream repo ships only in Mistral-native format (no config.json), so it cannot be loaded by AutoModel.from_pretrained. This repo makes it transformers-loadable, which also unblocks quantization toolchains (llm-compressor, etc.).

Architecture

Composed `Mistral3` = DeepSeek-V3 text backbone (MLA + shared-expert MoE: 128 routed experts, 4 active/token, 1 shared) + Pixtral vision. Built by mapping native tensors → HF names, fusing per-expert weights into the fused MoE layout, and dequantizing FP8→BF16.

✅ Verified / ⚠️ Not yet verified (tested 2026-07-04 on 1× B200, CUDA 13)

—✅ Structurally exact: strict key+shape match (801 tensors) against a real Mistral3(deepseek_v3+pixtral); loads cleanly in transformers.
—⚠️ End-to-end generation on a single B200 not confirmed: BF16 is ~238 GB, larger than one B200's 183 GB VRAM, so it needs multi-GPU (the base model uses tensor-parallel 4) or CPU offload. Short-context generation via device-map offload is functional but slow.
—⚠️ Vision converted (Pixtral tensors + preprocessor_config.json) but not runtime-validated.
—⚠️ RoPE approximate: source carries both yarn and llama_4_scaling; only YARN is mapped. Long-context (>8k) not guaranteed. Short context unaffected.

Provenance & precision

FP8 e4m3 (per-tensor scale) → BF16 (lossless upcast). Note these weights were already 8-bit in the source, not full-precision training weights.

Intended usage (mirrors the base model)

Tokenizer is Tekken (tekken.json, via mistral_common); Mistral instruct chat format (no manual [INST]); base recommends temperature 1.0, reasoning-parser mistral, context ≤ 200k. Example:

bash

vllm serve sahilchachra/Leanstral-1.5-119B-A6B-BF16 \
  --tokenizer-mode mistral --tensor-parallel-size 4 \
  --reasoning-parser mistral --tool-call-parser mistral --enable-auto-tool-choice

For lower memory / single-GPU, prefer the NVFP4 build: `sahilchachra/Leanstral-1.5-119B-A6B-NVFP4`.

Files: config.json, weights + index, tekken.json, params.json, chat_template.jinja.

License