sayyidfareed/DeepSeek-V4.1-Flash-Next-DGX-Spark-512K

DeepSeek-V4.1-Flash-Next-DGX-Spark-512K

A self-contained, 512K-qualified DeepSeek-V4.1-Flash deployment for one 128 GB NVIDIA DGX Spark.

On an ASUS GX10, this fixed K154 release handled a 524,293-token prompt, returned all five buried values, and generated 44 more tokens. Warm short coding prompts decoded at a 27.79 tok/s median with 1.32 s median TTFT.

The released DeepSeek-V4.1-Flash checkpoint is about 510 GB—far beyond the memory of one Spark. Khaled Bakeer (0xBakeer) built and published the single-Spark runtime that made this possible: Engram lookups from NVMe, a resident expert arena, CB3 experts, Triton kernels, DSpark speculative decoding, and the benchmark-driven pruning path.

This release takes the next deployment step. It permanently packages 154 routed experts in each of the 40 main layers, removes the original main routed-expert tensors, and verifies every retained shard before serving. There is no routing trace to supply, no machine-local warm-start artifact, and no expert re-quantization at startup.

This repository contains the complete self-contained model payload: retained DeepSeek weights, the 40-layer K154 CB3-v2 expert pack, tokenizer, encoding, configuration, DSpark drafter, manifest, and digests. The matching runtime is `sayyidfareed/deepseek-v4.1-flash-next-dgx-spark-512k`, tag v1.0.0.

Measured on the GX10

Hardware: NVIDIA GX10 / GB10 with 121.63 GiB usable unified memory and local NVMe. The model service was the only large workload on the box. Short-speed numbers are the second identical five-language pass after graph warmup; the first pass contained one first-shape compilation outlier and is retained in the qualification evidence rather than silently mixed into the warm result.

What K154 means

DeepSeek has 384 routed expert blocks in each of 40 main layers. For every token, the router normally picks six. The complete checkpoint contains 15,360 of those expert blocks, which is too much to keep in a Spark's unified memory.

K154 means this release keeps a fixed set of 154 experts per layer: 6,160 total. Those experts are stored in a compact three-bit-per-row CB3-v2 format and loaded into the 89.16 GB resident arena. The router is restricted to that exact set, so serving does not fall back to missing source experts.

This is not a retrained model. Dense weights, Engram tables, tokenizer, configuration, and DSpark drafter come from DeepSeek's checkpoint. The tradeoff is deliberate: fixed-expert pruning makes a 748B-class model practical on one 128 GB machine, but it can change quality outside the coding and reasoning workloads used to choose and validate the set.

Why a custom package was necessary

The original checkpoint's two Engram tables account for about 203 GB and stay on NVMe. The routed experts account for another 288.78 GB. Keeping the original routed tensors alongside a resident packed copy would make a supposedly publishable model hundreds of gigabytes larger than necessary.

This repository therefore contains:

• —221,508,192,600 bytes of retained non-main-routed tensors;
• —89,041,469,440 bytes of K154 CB3-v2 routed experts;
• —310,549,662,040 tensor bytes in total, about 289.22 GiB;
• —no original main routed-expert tensors.

The 40 rewritten base shards and 40 CB3 layer shards are authenticated by the manifest. Startup fails before the API port binds if a shard, tensor layout, selection, or source identity differs.

Quickstart

Requirements:

• —DGX Spark, ASUS GX10, or compatible GB10 system with 128 GB unified memory;
• —NVIDIA container runtime and Docker;
• —at least 330 GB free on fast local NVMe;
• —Hugging Face CLI for the roughly 311 GB model download;
• —about 17 minutes for the tested cold integrity check and model load.

hf download sayyidfareed/DeepSeek-V4.1-Flash-Next-DGX-Spark-512K \
  --local-dir /models/DeepSeek-V4.1-Flash-Next-DGX-Spark-512K

git clone --branch v1.0.0 \
  https://github.com/sayyidfareed/deepseek-v4.1-flash-next-dgx-spark-512k.git
cd deepseek-v4.1-flash-next-dgx-spark-512k

MODEL_PARENT=/models docker compose -f compose.k154.yaml up --build

The server is ready when the log says serving DeepSeek-V4.1-Flash-Next-DGX-Spark-512K. It exposes an OpenAI-compatible API on 127.0.0.1:8000 by default:

curl http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "DeepSeek-V4.1-Flash-Next-DGX-Spark-512K",
    "messages": [{"role":"user","content":"Write a lock-free ring buffer in Rust."}],
    "temperature": 0,
    "max_tokens": 1024,
    "stream": true,
    "tool_choice": "none"
  }'

The API has no built-in authentication. Keep it on loopback or place an authenticated proxy and trusted-network boundary in front of it.

The qualified serving profile

The immutable launcher supplies these values and refuses routing-trace or arbitrary extra flags. Changing the runtime, expert set, memory floor, or kernels creates a new profile that needs fresh qualification.

The qualified container used Ubuntu 24.04, Python 3.12.3, CUDA 13.0, PyTorch 2.13.0+cu130, Triton 3.7.1, Transformers 5.17.0, Tokenizers 0.23.2, Safetensors 0.8.0, NumPy 2.3.5, and SymPy 1.14.0. The Dockerfile pins the CUDA base by digest and the Python package versions used by the release.

Short-context quality check

The static package was rerun through the complete 164-problem HumanEval+ Mini generation set and the frozen executable coding microbenchmark. Generated programs were executed in network-disabled containers. The model produced:

• —HumanEval: 150/164;
• —HumanEval+ Mini: 142/164;
• —executable microbenchmark: 29/34.

The 29/34 microbenchmark result exactly matches the earlier accepted K154 runtime. This is a bounded coding qualification, not a general benchmark claim.

Reproduce the 512K test

Start with a small smoke request:

python3 bench/long_context_probe.py \
  --base-url http://127.0.0.1:8000 \
  --target-tokens 5000 \
  --tokenizer-json /models/DeepSeek-V4.1-Flash-Next-DGX-Spark-512K/tokenizer.json \
  --output smoke-5k.json

The exact accepted run takes about one hour on the tested machine:

python3 bench/long_context_probe.py \
  --base-url http://127.0.0.1:8000 \
  --target-tokens 522744 \
  --max-output-tokens 128 \
  --timeout 7200 \
  --tokenizer-json /models/DeepSeek-V4.1-Flash-Next-DGX-Spark-512K/tokenizer.json \
  --expected-prompt-tokens 524293 \
  --prompt-token-tolerance 0 \
  --expected-prompt-sha256 af9b0c94b91caf0fd53f6bc324236d3d94d9070740b88b66e7f25cf2c74e2c54 \
  --expected-completion-tokens 44 \
  --min-prefill-tok-s 145 \
  --min-decode-tok-s 4 \
  --output reproduction-512k.json

The probe generates deterministic varied archive records and inserts five unique values at roughly 5%, 25%, 50%, 75%, and 95%. It is a retrieval and operational-stability test, not a comprehensive long-context reasoning test.

Operational limits

• —512K is qualified; 1M is not. The engine reserves a 1,048,576-token capacity, but the successful evidence stops at 524,337 total tokens.
• —This is a single-sequence profile. It is tuned for one large request, not aggregate multi-user throughput.
• —Do not co-locate another large model. CPU, page cache, GPU allocations, and KV cache share the same 128 GB pool.
• —Use fast local NVMe. The 203 GB Engram lookup tables remain on storage and are accessed during inference.
• —Expect a long cold start. Integrity hashing and deterministic loading are intentional; warm-request TTFT does not include them.
• —The fixed expert set is workload-sensitive. Re-evaluate quality before using it as a specialist outside the validated coding/reasoning profile.
• —Multimodal input, multi-user concurrency, and 1M-context operation were not part of this qualification.

Reproducibility

Credits and licenses

• —DeepSeek created DeepSeek-V4.1-Flash and released the source checkpoint.
• —Khaled Bakeer (0xBakeer) created the upstream single-DGX-Spark runtime and the core Engram, expert-residency, CB3, kernel, memory, and optimization work this release materially depends on. See `0xBakeer/deepseek-v41-flash-spark` and CREDITS.md.
• —NVIDIA provides the DGX Spark / GB10 and CUDA stack; OpenAI Triton provides the kernel language and JIT.

The runtime code is MIT licensed. DeepSeek-derived weights and reference files retain their upstream license. See NOTICE before redistribution.