trjxter/Qwimi3.5-9B-Kimik2.6-Opus-Distill-MLX-8Bit

Qwimi3.5-9B KimiK2.6 Opus Distill MLX 8-bit

This is the MLX 8-bit conversion of:

`trjxter/Qwimi3.5-9B-Kimik2.6-Opus-Distill-BF16`

It is intended for Apple Silicon / MLX users who want to run the Qwimi3.5-9B reasoning fine-tune locally.

This repo is an MLX-format conversion/quantization of the merged BF16 model. The training was done separately using Unsloth + LoRA SFT, then merged to BF16 before MLX conversion.

Model Summary

Quantization Notes

This repository contains the 8-bit MLX version.

The 8-bit version is the highest-quality MLX quantization target in this set. It uses more storage and memory than 4-bit or 6-bit, but should generally preserve the most behavior from the merged BF16 source model.

For other formats, see the main release family:

• —BF16: `trjxter/Qwimi3.5-9B-Kimik2.6-Opus-Distill-BF16`
• —GGUF: `trjxter/Qwimi3.5-9B-Kimik2.6-Opus-Distill-GGUF`

Install

pip install -U mlx-lm

Basic Usage

mlx_lm.generate \
  --model trjxter/Qwimi3.5-9B-Kimik2.6-Opus-Distill-MLX-8bit \
  --prompt "Explain RS-LoRA in simple terms."

For longer generations:

mlx_lm.generate \
  --model trjxter/Qwimi3.5-9B-Kimik2.6-Opus-Distill-MLX-8bit \
  --prompt "Solve this step by step: If a company grows revenue from 120M to 180M over 3 years, what is the CAGR?" \
  --max-tokens 1024

Chat Template / Prompting

This model was trained with Qwen-style chat formatting and response-only SFT. For best results, use an instruction/chat style prompt.

Example:

You are a helpful reasoning assistant.

Question:
Explain why RS-LoRA can help at higher LoRA ranks.

The model was trained on reasoning-style outputs where assistant responses often contain:

<think>
reasoning trace
</think>
final answer

Depending on your inference settings and prompt, the model may produce explicit reasoning traces. For end-user applications, you may want to post-process or hide internal reasoning and show only the final answer.

Training Data

The merged BF16 source model was trained on a combined reasoning/distillation mix from:

1. 1.trjxter/Kimi-K2.6-Reasoning-3300x-WandB
2. 2.Jackrong/Qwen3.5-reasoning-700x
3. 3.Jackrong/Claude-opus-4.6-TraceInversion-9000x

After formatting and 16k-token filtering, the final usable dataset size was:

12,366 usable examples
12,000 train examples
366 held-out eval examples
200 trainer eval examples

The curation process normalized mixed schemas into a shared Qwen chat-template SFT format.

Training Configuration

The original fine-tune used:

Base model: unsloth/Qwen3.5-9B
Training: LoRA SFT
Base loading: 8-bit
LoRA rank: 128
LoRA alpha: 128
RS-LoRA: enabled
Max sequence length: 16,384
Gradient checkpointing: unsloth
Batch size: 2
Gradient accumulation: 8
Effective batch size: 16
Epochs: 1
Optimizer: adamw_8bit
Learning rate: 2e-5
Warmup steps: 30
Save steps: 200
Eval steps: 200

Expected optimizer steps:

12,000 / 16 = 750 optimizer steps

Training Results

Final trainer metrics from the source fine-tune:

{
    "train_runtime": 15728.7981,
    "train_samples_per_second": 0.763,
    "train_steps_per_second": 0.048,
    "total_flos": 1.4528477736160543e+18,
    "train_loss": 0.5516796213785807,
    "epoch": 1.0,
}

Loss checkpoints:

Artifact Lineage

flowchart TD
    A[unsloth/Qwen3.5-9B] --> B[LoRA SFT]
    B --> C[Qwimi LoRA adapter]
    C --> D[Merged BF16 model]
    D --> E[MLX 8-bit conversion]

Intended Use

This model is intended for:

• —general reasoning experiments,
• —local Apple Silicon inference,
• —MLX benchmarking,
• —educational fine-tuning research,
• —testing Qwen-style reasoning distillation behavior.

It is not intended for high-stakes medical, legal, financial, or safety-critical decision-making without expert review.

Limitations

• —This is a community fine-tune and should be evaluated before production use.
• —The model may produce incorrect reasoning or confident wrong answers.
• —The model may generate verbose reasoning traces depending on prompts.
• —MLX quantized behavior can differ from BF16 or GGUF behavior.
• —This README reports the training metrics of the BF16 source model, not a separate full eval of this exact MLX quant.

Citation / Attribution

Base model and tooling credit goes to the Qwen, Unsloth, Hugging Face, and MLX communities.

Source fine-tune:

trjxter/Qwimi3.5-9B-Kimik2.6-Opus-Distill-BF16

MLX conversion:

trjxter/Qwimi3.5-9B-Kimik2.6-Opus-Distill-MLX-8bit

Notes

This MLX quant was created to make the Qwimi3.5-9B run easier to use for Apple Silicon users.