pnoid/Mistral-Small-3.2-24B-OpenThoughts-Distill-v1-mlx-4bit

Mistral-Small-3.2-24B-OpenThoughts-Distill-v1 (MLX 4bit)

A LoRA fine-tune of Mistral Small 3.2 24B Instruct on the OpenThoughts-114k reasoning dataset, intended to strengthen structured chain-of-thought reasoning patterns. Released in the MLX 4-bit format for Apple Silicon.

GGUF quantizations will be released as a separate companion repository — see the Roadmap below.

Model Details

Training

LoRA fine-tune with mlx-lm on an Apple M4 Pro (64 GB unified memory).

Reproducible training command

mlx_lm.lora \
  --model mlx-community/Mistral-Small-3.2-24B-Instruct-2506-4bit \
  --train --data ./data --iters 1500 \
  --batch-size 1 --grad-accumulation-steps 2 \
  --num-layers 4 --grad-checkpoint --mask-prompt \
  --learning-rate 1e-5 --max-seq-length 2048 \
  --save-every 250 --steps-per-eval 250 \
  --adapter-path ./adapters

After training, the resulting LoRA adapter was merged into the base via mlx_lm.fuse.

Dataset

[`open-thoughts/OpenThoughts-114k`](https://huggingface.co/datasets/open-thoughts/OpenThoughts-114k) — 113,957 reasoning trace examples, generated by DeepSeek-R1, released under Apache 2.0.

The ShareGPT-style schema (system + conversations[from/value]) was converted to mlx-lm's chat-template-compatible {"messages": […]} JSONL format:

Caveat — Sequence Truncation

OpenThoughts contains very long reasoning chains (single examples up to ~23k tokens). To fit the 64 GB Mac memory budget, max_seq_length was set to 2048. As a result, the model is trained on the first 2048 tokens of every example and the long tails of longer reasoning chains were dropped.

Practical implication: the model has been exposed to reasoning structures up to 2048 tokens. When generating responses substantially longer than that on its own, quality may degrade earlier than the chosen max_tokens cap. A v2 with a higher sequence length is on the roadmap.

Usage (MLX)

from mlx_lm import load, generate

model, tokenizer = load(
    "pnoid/Mistral-Small-3.2-24B-OpenThoughts-Distill-v1-mlx-4bit"
)

messages = [
    {"role": "user",
     "content": "Three friends share 24 cookies equally. "
                "One friend gives 3 cookies to each of the others. "
                "How many does each friend have now? "
                "Reason step by step."}
]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
response = generate(
    model, tokenizer, prompt=prompt, max_tokens=1024, verbose=True
)

CLI:

mlx_lm.generate \
  --model pnoid/Mistral-Small-3.2-24B-OpenThoughts-Distill-v1-mlx-4bit \
  --prompt "Your reasoning question here" \
  --max-tokens 1024

Reasoning Style — Observed Differences

This v1 release is honestly framed as a stylistic distillation, not a capability leap. The goal was to transfer R1-style explicit-reasoning patterns onto a strong base. Side-by-side probes on classic reasoning prompts (cookie-share math, bat-and-ball Cognitive Reflection Test, sister-brother counting trap) gave the following picture:

Pattern transfer (intended outcome — present)

The fine-tuned model exhibits R1-signature self-verification language that the base model does not, e.g.:

"Let me rephrase the information to make sure I get it right…" "Wait, let me reconsider…" "Actually, let me check…"

This is the visible signature of the OpenThoughts-114k (R1-generated) reasoning data.

Capability gain (honest note — not demonstrated)

On the probes above, the base Mistral Small 3.2 24B Instruct already produces correct, structured, didactic answers — including a spontaneous "Common Pitfall" warning on the bat-and-ball trap. This v1 fine-tune did not measurably improve accuracy on those particular probes.

Likely contributing factors:

• —The base is already strong at structured reasoning out-of-the-box.
• —Training was on a 4-bit quantized base (gradients against quantized weights).
• —The 2048-token training truncation means the model has been exposed to starts of long reasoning chains but not to their convergence. On long traps (sister-brother), the fine-tune showed more self-questioning loops but did not converge faster than the base.

What this release is useful for

• —An MLX-format, Apache-2.0, openly-licensed-chain model that visibly exhibits R1-style explicit reasoning on Apple Silicon.
• —A baseline for our own iteration: a v2 trained on a higher-precision base, longer sequence length, and more iterations is on the roadmap and is where a measurable capability shift is expected.

License Chain

The entire supply chain is Apache 2.0. No proprietary LLM outputs (e.g. OpenAI, Anthropic) were used either in dataset generation or in training.

Roadmap

• —GGUF quantizations (at minimum Q4KM and Q8_0) released as a sibling …-GGUF repository
• —v2 with higher sequence length and more iterations once a fp16/bf16-capable host is available
• —Public evaluation on standard reasoning benchmarks (GSM8K, MATH, MMLU-Pro)

Limitations

• —Sequence truncation during training at 2048 tokens may cause earlier-than-ideal stopping behaviour on very long self-generated reasoning chains.
• —4-bit quantized base before LoRA: gradients are computed against the quantized weights. Quality is solid for distillation but a fp16/bf16-based v2 would likely be stronger.
• —Language coverage: the training dataset is predominantly English; multilingual reasoning quality may vary by language.
• —Multimodality: the base architecture is Mistral3 (vision + text). This LoRA is purely text-only training; vision capabilities are inherited unchanged from the base and have not been fine-tuned here.

Attribution & Acknowledgements

• —[Mistral AI](https://mistral.ai/) — for releasing an excellent multilingual base model under Apache 2.0.
• —[OpenThoughts team](https://huggingface.co/open-thoughts) — for assembling and releasing the high-quality reasoning distillation dataset.
• —[DeepSeek](https://www.deepseek.com/) — for the R1 model whose generations form the reasoning traces inside OpenThoughts-114k.
• —[mlx-community](https://huggingface.co/mlx-community) — for the MLX 4-bit quantization of the base model used as the LoRA host.
• —[Apple ML Research](https://github.com/ml-explore/mlx-examples) — for mlx-lm, which made training on a 64 GB Mac feasible.

Citation

@misc{pnoid_mistral_openthoughts_distill_v1_mlx_4bit_2026,
  author       = {pnoid},
  title        = {Mistral-Small-3.2-24B-OpenThoughts-Distill-v1 (MLX 4-bit)},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/pnoid/Mistral-Small-3.2-24B-OpenThoughts-Distill-v1-mlx-4bit}}
}

When using this model, please also cite the base model and the dataset (see their respective model/dataset cards for canonical citation formats).

Trained June 2026 on an Apple M4 Pro (64 GB unified memory).