Modelpublic

jedisct1/MiMo-V2.6-Flash-RL-GGUF-100K

sourceHugging Facemitupdated 4d agoView on Hugging Face

1likes117downloads

Model Card

MiMo V2.6 Flash RL GGUF, 100K

This is a text-only GGUF release of XiaomiMiMo/MiMo-V2.6-Flash-RL for local coding, tool use, and long technical work.

It is made for recent llama.cpp builds and LM Studio on high-memory machines. The tested setup is a 128 GB Apple Silicon Mac with a 100,000-token total context limit.

The model is split into four files and takes about 100 GiB on disk. Keep all four GGUF files in the same directory and open the first shard.

What it is good at

—Writing, reading, and fixing code.
—Working through repositories with file and command tools.
—OpenAI-compatible tool calling.
—Long English prompts, large source files, and technical documents.
—Tasks that need several tool calls and a final answer.

English coding and agent work received the most testing. The original model also supports Chinese, but Chinese quality was not a focus for this release.

This GGUF is text only. It does not include working image, audio, or video input.

Download

hf download jedisct1/MiMo-V2.6-Flash-RL-GGUF-100K \
  --local-dir MiMo-V2.6-Flash-RL-GGUF-100K

You can check the download from inside that directory:

shasum -a 256 -c SHA256SUMS

Run with llama.cpp

Use llama.cpp commit ff0dbb975e93a9a2899efa34bdd32d1c5cfbc183, or a newer build with the MiMo2 sliding-window fix and MXFP4 support. This is the tested starting point for a 128 GB Apple Silicon machine:

MODEL_DIR=MiMo-V2.6-Flash-RL-GGUF-100K

llama-server \
  --model "$MODEL_DIR/MiMo-V2.6-Flash-RL-Mixed-IQ3XXS-IQ2XS-Q8-00001-of-00004.gguf" \
  --host 127.0.0.1 \
  --port 8080 \
  --ctx-size 100000 \
  --parallel 1 \
  --kv-unified-per-slot 100000 \
  --fit on \
  --fit-target 8192 \
  --fit-ctx 100000 \
  --load-mode none \
  --lazy-mode off \
  --no-repack \
  --flash-attn on \
  --cache-type-k q8_0 \
  --cache-type-v q8_0 \
  --batch-size 512 \
  --ubatch-size 512 \
  --cache-ram 0 \
  --no-cache-idle-slots \
  --no-warmup \
  --jinja \
  --reasoning auto \
  --reasoning-format deepseek \
  --reasoning-preserve \
  --spec-type none

The server exposes an OpenAI-compatible API at http://127.0.0.1:8080/v1.

Do not add --gpu-layers all to this command. It conflicts with llama.cpp's automatic fit mode.

Run with LM Studio

Download all four shards into one directory, then add that directory to LM Studio and select the first shard.

Use these settings as the starting point on a 128 GB Apple Silicon Mac:

Setting	Value
Context length	`100000`
GPU offload	Maximum or all layers
Flash attention	On
K cache	`q8_0`
V cache	`q8_0`
Memory mapping	Off
MTP speculative decoding	Off

LM Studio may display 100096 after loading. That is its aligned physical allocation for a requested 100,000-token context. Keep the client limit at 100000.

Choose the model in LM Studio, apply the settings above in the load panel, then load it. The current lms load command cannot express every tested setting, so the GUI load panel is the safer option for long prompts.

Context and memory

The 100,000-token limit includes the prompt, tool results, conversation history, reasoning, and generated answer. Leave enough room for the answer instead of filling the whole window with the initial prompt.

Long prompts get increasingly expensive as they grow. The model can hold more context in theory, but 100,000 is the practical limit recommended for this release.

Close other memory-heavy applications before loading it. This full-offload setup needs a 128 GB Mac. A smaller-memory system may not fit the model at all; where CPU offload is available, expect it to be much slower.

Tool calling

The tool template is built into the GGUF. Do not replace it with a generic Qwen template.

Send normal OpenAI-compatible tool schemas from your client. If the first action definitely needs a tool, using "tool_choice": "required" for that first request is more reliable. Return to automatic tool choice after the tool call.

Keep confirmation and result checks around destructive or unattended work.

Known limits

—This release is text only.
—Full-context prompts are useful, but they are not instant.
—MTP is kept off because it does not speed up prompt loading and was not part of the reliable tool-use setup.
—Tool calling can still miss a step or return extra commentary.
—The tested memory settings are specific to a 128 GB Apple Silicon Mac.

License

The upstream model card declares the MIT license. This GGUF release uses the same license metadata. Review the upstream model card for the original model's description and terms.