xautonomics/DeepSeek-V4-Flash-Vision-Exp-UD-IQ4_XS.LAYR.GGUF

DeepSeek-V4-Flash-Vision-Exp-UD-IQ4_XS.LAYR.GGUF

A per-layer GGUF library of Unsloth's unsloth/DeepSeek-V4-Flash-Vision-Exp-GGUF (UD-IQ4_XS). These are the same quantized weights, sliced once into one file per transformer block plus the shared parts, so any contiguous layer window [A, B) can be assembled at load time. It's built for dreamcatcher, xAutonomics' fork of ik_llama.cpp, which serves a library either in a single process or split across hosts.

Support status

This quant first, then the architecture status quoted from the dreamcatcher tree (as of `c15270d0`; the linked document is authoritative):

deepseek4 — this UD-IQ4_XS library has not been run on dreamcatcher yet. It is sliced from the same pinned Unsloth revision (37044a3c, 1328 tensors, no exp_probs_b_vl vision tensors) as the UD-Q4KXL build the architecture was measured on, and is published so the IQ4 path can be measured, including on Vulkan RADV. This card will be updated when it is.

Architecture status, measured on UD-Q4_K_XL at that revision: monolith verified on CUDA (1328/1328 tensor hashes, full 43-layer forward pass byte-identical). Library windows fixed (meta#92): ring token-identical 64/64 on CUDA. llama-server --model-dir is available (meta#90; not yet measured on this architecture). Vulkan: degenerate on NVIDIA Vulkan (coopmat1), not quant-related (meta#96); RDNA3/ANV not measured.

Only model files from huggingface.co/xautonomics are supported by dreamcatcher. Other GGUFs, including libraries you slice yourself, may load but are unsupported. Do not rename the files: the loader expects blk-NNNNN.gguf / parts-*.gguf / manifest.json exactly.

Layout

blk-00000.gguf … blk-00042.gguf   one transformer block each
parts-embd.gguf                          token embedding (every stage loads it)
parts-output.gguf                        output norm + head (every stage loads it)
parts-other.gguf                         remaining non-block tensors (every stage loads it)
manifest.json                            file list, sizes, per-file + per-tensor hashes, source provenance

A stage that serves layers [A, B) needs blk-A … blk-(B-1) plus the parts-* files and manifest.json.

Use

Single process (the whole model on one host):

llama-server --model-dir /models/DeepSeek-V4-Flash-Vision-Exp-UD-IQ4_XS.LAYR.GGUF

Multi-host ring (stages exchange hidden states over TCP; each host needs low-latency, direct node-to-node links — see the dreamcatcher README):

llama-stage-runner --role tail --listen 8081 --model-dir /models/DeepSeek-V4-Flash-Vision-Exp-UD-IQ4_XS.LAYR.GGUF --layers 24,43
llama-stage-runner --role head --connect <tail-host>:8081 --model-dir /models/DeepSeek-V4-Flash-Vision-Exp-UD-IQ4_XS.LAYR.GGUF --layers 0,24

Plan a split for your hosts (weights-only screen; the proof is a survived prefill), and verify the download before launching:

python3 -m layer_distribution.plan /models/DeepSeek-V4-Flash-Vision-Exp-UD-IQ4_XS.LAYR.GGUF/manifest.json --node a:16GiB:200GiB:8080 --node b:24GiB:200GiB:8081
python3 -c "from layer_distribution import verify; r = verify('/models/DeepSeek-V4-Flash-Vision-Exp-UD-IQ4_XS.LAYR.GGUF', '/models/DeepSeek-V4-Flash-Vision-Exp-UD-IQ4_XS.LAYR.GGUF/manifest.json'); print(r.passed, r.hash_verified)"

Provenance and license

Weights are byte-for-byte the tensors of unsloth/DeepSeek-V4-Flash-Vision-Exp-GGUF at revision 37044a3cff5ed45e3832f3e833bf1cff91f7f168 (Unsloth's dynamic quantization of deepseek-ai/DeepSeek-V4-Flash-Vision-Exp). Source shard names and sizes are recorded in manifest.json (source.shards[], source.content_hash). License follows the base model: mit, see LICENSE. Credit to the base-model authors and to Unsloth for the quantization.

About

Published by xAutonomics for dreamcatcher.