GreenBitAI/DeepSeek-V4.1-Flash-4bit-paged

DeepSeek-V4.1-Flash-4bit-paged

Expert-paged build of pipenetwork/DeepSeek-V4.1-Flash-MLX-mixed-4_8bit. The weights that are read a fraction at a time live in their own containers, so a machine loads what it needs rather than all of it.

Total 406.10 GiB. The bytes moved into containers of their own; they were not copied.

These containers are not a format mlx-lm reads. The model runs on `gbx_lm`, a single signed binary for Apple Silicon; there is nothing to pip install.

Requirements

Memory is not a fixed figure for a paged build, and that is the point of one: it fills what fits and streams the rest from disk. On a 512 GB Mac Studio with room to spare this model settles at about 306 GB resident. A smaller machine holds less and reads more from disk -- slower, but it runs.

How much slower depends on how far the machine is from holding the experts, and on how fast its disk is. Each token routes to a few experts; the ones already in memory cost nothing to reach, and the ones that are not have to be read before that token can finish. A machine holding most of them waits rarely, one holding few waits often. We have not measured this across machine sizes and will not guess a figure: what we can say is that the model answers either way, and that the wait is the SSD's, not the model's.

Install

curl -fL -o gbx_lm-darwin-arm64.tar.gz 'https://github.com/GreenBitAI/gbx-lm/releases/latest/download/gbx_lm-darwin-arm64.tar.gz' \
  && tar -xzf gbx_lm-darwin-arm64.tar.gz gbx_lm \
  && mkdir -p "$HOME/.local/bin" \
  && mv gbx_lm "$HOME/.local/bin/gbx_lm" \
  && chmod +x "$HOME/.local/bin/gbx_lm"

gbx_lm -h

command not found means $HOME/.local/bin is not on your PATH: add it, or call the binary by its full path. The build is signed with a Developer ID and notarised, so macOS runs it without the usual detour for a downloaded binary.

Run

gbx_lm --model GreenBitAI/DeepSeek-V4.1-Flash-4bit-paged

That serves an OpenAI-compatible API on port 11688, which is its default. The weights download on first use into ~/.libra/cache/models; set HF_HOME to put them elsewhere, and HF_TOKEN if you meet the Hub's rate limits for anonymous downloads.

curl http://127.0.0.1:11688/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"GreenBitAI/DeepSeek-V4.1-Flash-4bit-paged","messages":[{"role":"user","content":"Hello"}]}'

Where the weights fit they are filled from experts.bin and the model runs the stock path at stock speed; where they do not, they stream from disk. Reading the machine decides that, not a flag.

To override that: GBX_PAGING=off holds the experts resident, GBX_ENGRAM=off holds the engram tables resident.

The directory also carries DeepSeek's own DSpark draft head, off unless asked for: GBX_DEEPSEEK_MTP=on decodes speculatively against it. It is converted from deepseek-ai/DeepSeek-V4.1-Flash and keeps that licence; no published MLX build carries it, which is why it is here.

Checked at build time, while the source checkpoint was still there to compare against:

• —PASS bit-identical logits — exact on 5 prompt(s) to 160 tokens; 1 longer differ by at most 15, same greedy token throughout
• —PASS layer-wise vs resident — 40 layers x 2 draws exact, 377.86 GiB peak for this gate

Quantization, tokenizer, chat template and licence are unchanged from pipenetwork/DeepSeek-V4.1-Flash-MLX-mixed-4_8bit.

Coding agents

The server speaks three wire protocols on the same port, so the tools that expect a hosted API can be pointed at this one:

Codex -- a provider in ~/.codex/config.toml:

[model_providers.gbx]
name = "gbx-lm"
base_url = "http://127.0.0.1:11688/v1"
wire_api = "responses"

and a profile in ~/.codex/gbx.config.toml:

model_provider = "gbx"
model = "GreenBitAI/DeepSeek-V4.1-Flash-4bit-paged"
model_context_window = 1048576

Claude Code -- ~/.claude/gbx.settings.json:

{
  "env": {
    "ANTHROPIC_BASE_URL": "http://127.0.0.1:11688",
    "ANTHROPIC_AUTH_TOKEN": "local",
    "ANTHROPIC_MODEL": "GreenBitAI/DeepSeek-V4.1-Flash-4bit-paged",
    "ANTHROPIC_DEFAULT_HAIKU_MODEL": "GreenBitAI/DeepSeek-V4.1-Flash-4bit-paged"
  }
}

Both clients ask for a small model for their own background work, so every name in the settings has to be one this server is serving.

The draft head

The mtp/ folder carries the model's own multi-token prediction head, so speculative decoding works from this repository alone. It is off unless asked for:

GBX_DEEPSEEK_MTP=on gbx_lm --model GreenBitAI/DeepSeek-V4.1-Flash-4bit-paged
GBX_DEEPSEEK_MTP=on gbx_lm --model GreenBitAI/DeepSeek-V4.1-Flash-4bit-paged

Up to 2.3x, and it holds at a long context. Measured 2026-09-18 on a 512 GB Mac Studio (M3 Ultra) over HTTP, in an agent loop carrying tool definitions and a session cache, decode timed from the first token; the median request of each session:

It does not fade as the history grows: index_topk bounds attention during decode, so the backbone's cost per token is nearly flat and the head's saving is not swamped by a growing denominator. It wins most where the next token is guessable -- source code over prose. Every token it proposes is verified by the model itself, so the reply is the model's own either way; the head only saves passes over the weights.