Modelpublic

apetersson/DeepSeek-V4-Flash-0731-Abliterated-DS4-Headroom128

sourceHugging Facemitupdated 2mo agoView on Hugging Face

34likes6kdownloads

Model Card

DeepSeek-V4-Flash-0731 Abliterated — DS4 Headroom128

Experimental low-bit model-surgery derivative. This release is designed to run DeepSeek-V4-Flash-0731 with its real three-stage DSpark drafter on a 128 GB Apple Silicon machine. Structural validation has passed, but direct generation, quality, refusal-removal, DSpark-acceptance, throughput and peak memory measurements for this exact artifact are still pending.

This is a DS4-native mixed-precision GGUF conversion of `apetersson/DeepSeek-V4-Flash-0731-Abliterated-FP8`, which is itself a rank-1 abliterated derivative of `deepseek-ai/DeepSeek-V4-Flash-0731`. It remains a derivative of the official post-trained 0731 model: it is not a base model, was not gradient-finetuned, and is not claimed to be universally "uncensored."

What is included

File	Purpose	Size
`DeepSeek-V4-Flash-0731-Abliterated-DS4-Headroom128.gguf`	Authoritative 43-layer target model	86,720,111,552 bytes (80.76 GiB)
`DeepSeek-V4-Flash-0731-Abliterated-DS4-Headroom128-DSpark-support.gguf`	Three-stage DSpark speculative drafter; not standalone	5,989,114,272 bytes (5.58 GiB)
`BUILD_MANIFEST.json`	Complete source, profile, calibration, tool and command record	—
`PROVENANCE.md`	Compact provenance summary	—
`SHA256SUMS`	Integrity hashes for the release files	—

The two GGUF files total approximately 86.34 GiB. The profile deliberately spends precision on dense paths used for every token while aggressively compressing the much larger routed-expert bank, leaving practical unified- memory headroom for DS4 runtime state and KV cache on a 128 GB host.

Quantization profile

Main target model

Tensor class	Quantization/storage	Reasoning
Routed expert gate projections, all 43 layers	`IQ2_XXS`, importance-matrix calibrated	These dominate weight size; imatrix weighting preserves the most activation-relevant directions at very low bit rate.
Routed expert up projections, all 43 layers	`IQ2_XXS`, importance-matrix calibrated	Same size/quality trade-off as routed gate.
Routed expert down projections, all 43 layers	`Q2_K`, importance-matrix calibrated	Down projections directly write expert output back to the residual stream; `Q2_K` is the more conservative two-bit choice here.
Attention projections	`Q8_0`	Attention is a dense, per-token path, so errors are repeatedly exposed rather than diluted across sparsely selected experts.
Shared experts	`Q8_0`	The shared expert is active for every token and therefore receives substantially stronger protection than routed experts.
Vocabulary/output head	`Q8_0`	Protects final-logit fidelity and avoids compounding low-bit error at token selection.
Indexer `attn_q_b` tensors on layers 2,4,…,42	forced `F16`	The compressed-attention indexer is small enough to preserve and sensitive enough not to include in the generic attention Q8 rule.
Remaining control, normalization, routing and auxiliary tensors	template-declared `F16`/`F32`/`I32`	Small or numerically sensitive tensors are not forced into the routed-expert low-bit profile.

Observed main-GGUF tensor inventory from DS4 inspection:

GGUF type	Tensors	Described bytes
`IQ2_XXS`	86	44.34 GiB
`Q2_K`	43	28.22 GiB
`Q8_0`	345	6.15 GiB
`F16`	359	2.04 GiB
`F32`	492	<0.01 GiB
`I32`	3	0.01 GiB

The target GGUF is version 3, contains 1,328 tensors, describes approximately 284.33 billion logical parameters, and retains the checkpoint's declared 1,048,576-token training context. That context declaration is not yet a claim that one-million-token inference has been exercised with this artifact.

DSpark support model

This is the actual 0731 three-stage DSpark model (mtp.0–mtp.2), not the legacy single-stage MTP attachment. Each stage proposes positions in a block of five; the target model verifies the proposal and remains authoritative.

Parameter	Value
Stages	3
Proposal block size	5
Target layers	40, 41, 42
Markov rank	256
Noise token ID	128799
Routed gate/up	`IQ2_XXS` (6 tensors total)
Routed down	`Q2_K` (3 tensors total)
Dense projections	`Q8_0` (31 tensors)
Control/auxiliary tensors	7 `F16` + 34 `F32` tensors

The support GGUF contains 81 tensors and approximately 19.85 billion logical parameters. It must be loaded alongside the matching target GGUF.

Importance calibration

The routed-expert calibration source is the 0731-native matrix from `ox-ox/DeepSeek-V4-Flash-0731-GGUF`:

—Repository revision: 6d58a3a36030c3ccb969bb5759fc6ae08cd299f8
—File: imatrix/DeepSeek-V4-Flash-0731-chat-v2-routed-moe-ds4-1p5m.dat
—Size: 450,892,654 bytes
—SHA-256: 6fce7674df701de544e5d3351aab04e67602eddeafeb48cf70e77ebe47239eb4
—Coverage: exactly 129 target routed tensors (43 layers × gate/up/down), with exact name and vector-dimension validation

DS4's global --imatrix-strict was intentionally not used. That flag also requires calibration entries for F16 embeddings and Q8 dense tensors, while the published matrix is deliberately routed-only. The build driver instead performed an exact routed-name and vector-length coverage gate before invoking the quantizer, so every tensor that uses the matrix is covered without pretending that unrelated tensors were calibrated.

The published matrix has no native DSpark entries. For the support model, the calibration importer created deterministic aliases from the target features consumed by each stage:

—mtp.0 ← target layer 40
—mtp.1 ← target layer 41
—mtp.2 ← target layer 42

This extended matrix contains 138 entries and has SHA-256 689b446ed2e2657ebcb69a6516781ee0444e07641aeb6125233fb4e6fe7cbce3. It is an explicitly labeled target-layer proxy, not a fresh activation capture from the DSpark drafter.

Weight provenance and the metadata template

The only source of weight values was the local abliterated FP8 checkpoint. No tensor bytes were copied from ox-ox, antirez, another GGUF, or another quantized model. All 48 source weight shards (166,886,535,336 bytes) were verified against their recorded SHA-256 identities before conversion and guarded against mutation throughout the build.

The exact-0731 file from `antirez/deepseek-v4-gguf` was used only as a bounded metadata/tokenizer/tensor-order-and-shape template:

—Repository revision: 1cd7b564460821938add0475a60b942c409295e0
—Template LFS SHA-256: ca22ae2f838e14077c22bc1c1417b71b45b5e5a3687bd96c2ac6e17fdb6261c0
—Template Xet object: 7da16e1025c1b856490c29c341f4e467d15cb195389c70383dade5e6108799ac
—Verified 64 MiB header SHA-256: f0e1d5e8f3b008402aa6eb32cada3873dd926c8bf5e7d00d7788eec65f09dd6d
—Template weights used: no

Abliteration provenance

Before GGUF conversion, the native FP8 source underwent targeted refusal- direction projection:

—Method: rank-1 residual-writer projection with FP8 fixed-point requantization
—Direction source: `drowzeys/DeepSeek-V4-Flash-DSpark-Abliterated-Uncensored-1M-57toks`
—Direction revision: 85ae5bbaa7daf4da7b40e007a71a1c06b41fc053
—Direction SHA-256: 6e4d8a8f3aa9e21795faab2c5b14d29b019acdf2ddbfbd8238430458a5837fe0
—Attention strength: λ = 3.5
—Main layers edited: 10–42 inclusive
—DSpark attention output projections edited: all three stages
—Total edited tensors: 36
—Native FP8 fixed-point requantization iterations: 3

See the FP8 parent model card and its ABLITERATION_MANIFEST.json for per- tensor delta and FP8 round-trip measurements. The edit is experimental and may affect capabilities or behavior beyond refusal propensity.

Reproducibility

—Conversion tooling: `apetersson/deepseek-model-tools@19c1921`
—DS4 converter/runtime: `antirez/ds4@54b36ed`
—oMLX revision used by the build/import environment: 76352ed2363e42b2146243463875756e683a640d
—Exact profile SHA-256: 92edc9683ef0f9903dfd76feb7c0f762bbd218dcdf7331b2c9d0d8e9c6dbe0a7

BUILD_MANIFEST.json contains the complete profile, source shard identities, resolved calibration/template metadata, and exact converter command lines.

Installation note

The build manifest records the originally intended final operation, renameatx_np(RENAME_EXCL). The Samsung exFAT destination rejected that primitive with ENOTSUP (errno 45) after the complete hidden incoming tree had already passed checksum and DS4 inspection. Publication was recovered without requantization or recopying using the tested same-directory cooperative-sibling-lock+lstat+same-parent-renameat path from `deepseek-model-tools@37eb66a`. The rename is whole-tree atomic; exFAT does not provide a kernel-level no-replace directory primitive, so the fallback lock serializes cooperating installers and explicitly documents the residual race with an unrelated process that ignores that lock.

Running with DS4

Use the pinned DS4 revision above. This is a custom DeepSeek-V4/DSpark GGUF; compatibility with stock llama.cpp or unrelated GGUF runtimes is not claimed.

Greedy DSpark decoding:

bash

./ds4 \
  -m /path/to/DeepSeek-V4-Flash-0731-Abliterated-DS4-Headroom128.gguf \
  --mtp /path/to/DeepSeek-V4-Flash-0731-Abliterated-DS4-Headroom128-DSpark-support.gguf \
  --dspark --temp 0

--mtp supplies the support GGUF and --dspark selects the three-stage runtime. DSpark is opt-in, accelerates generation rather than prefill, and may be neutral or slower when acceptance is low. Sampled decoding does not use DSpark proposals in the pinned runtime.

Structural inspection:

bash

./ds4 --cpu --inspect \
  -m /path/to/DeepSeek-V4-Flash-0731-Abliterated-DS4-Headroom128.gguf \
  --mtp /path/to/DeepSeek-V4-Flash-0731-Abliterated-DS4-Headroom128-DSpark-support.gguf \
  --dspark-strict

Validation status

Completed on 2026-08-01:

—All release payloads passed their recorded SHA-256 checks immediately before atomic exposure.
—Main GGUF inspection: version 3, 62 metadata keys, 1,328 tensors, 43 layers, declared 1,048,576-token context.
—DSpark inspection: 3 stages, block size 5, target layers 40/41/42, 81 tensors.
—Support binding: 81 tensors, 0 missing, 0 invalid and 0 metadata errors.

Still required before making quality or performance claims for this exact artifact:

—successful load and deterministic generation with DSpark disabled;
—successful generation with DSpark enabled and lossless target agreement;
—proposal acceptance rate and accepted tokens per target step;
—prompt-processing and generation tokens/second;
—peak unified memory and practical context limits on the 128 GB M1 Ultra;
—capability/perplexity comparison against the abliterated FP8 source;
—refusal-removal comparison against the clean 0731 model and the FP8 parent.

Results obtained from another MLX or GGUF quant are useful context but are not reported here as measurements of this artifact.

Limitations and responsible use

Ultra-low-bit expert quantization can reduce reasoning, factuality, style fidelity and long-context robustness even when dense paths are protected. Abliteration can change behavior outside explicit refusal cases. Neither technique guarantees compliance, safety, correctness, neutrality or a particular response style. Apply appropriate access controls and evaluate the model for the intended deployment rather than relying on the word "abliterated."

License and attribution

The upstream repository and weights are MIT licensed. This derivative retains that license. Credit DeepSeek-AI for the original model, drowzeys/keys for the refusal direction, ox-ox for the 0731 routed-expert importance matrix, antirez for DS4 and the exact-0731 metadata recipe, and apetersson for this conversion and release tooling.