DevelopingDad/Qwen3.8-27B-Uncensored-NVFP4-MSE-DGX-Spark

Qwen3.8-27B-Uncensored-NVFP4-MSE-DGX-Spark

MSE-calibrated ModelOpt mixed-precision NVFP4/FP8 derivative of `orcarouter/Qwen3.8-27B-Uncensored`, optimized and validated for local inference on one NVIDIA DGX Spark.

This is the target-model weight repository. The reported speed result requires the complete serving stack:

• —SGLang on the DGX Spark GB10
• —an external calibrated DFlash2 draft model
• —speculative depth 8
• —BF16 KV cache and BF16 Mamba state

The DFlash2 draft is linked below but is not included in these weights.

Safety warning: the parent checkpoint is abliterated/refusal-removed. It may comply with harmful, unethical, or illegal requests. This repository is intended for controlled research and local evaluation. Add access controls, moderation, and abuse prevention before any user-facing deployment.

What is in this repository

The weights were produced from the immutable parent revision `9878936be9458522b5aeed0e13476bb8426f57f0` with NVIDIA Model Optimizer (ModelOpt) using its Qwen3.5 dense mixed-precision PTQ recipe and MSE calibration.

This is a quantized representation of the same OrcaRouter parent model, not a newly trained model. It is not a pure all-NVFP4 checkpoint.

The export contains FP8-KV metadata, but the validated production recipe deliberately uses BF16 KV and BF16 Mamba state. The headline benchmark does not use FP8 KV.

Integrity manifests:

• —`WEIGHTS_SHA256SUMS` covers the three published safetensors shards.
• —`SOURCE_ARTIFACT_SHA256SUMS` covers every unchanged file from the original C-MSE export. It intentionally excludes this repository's rewritten model card and added documentation.

Why the MSE artifact

The accompanying ModelOpt study produced both max-calibrated and MSE-calibrated exports. Their older, narrow quality rubric tied. In the later production gate, the max-calibrated artifact reached 33.88 tok/s but answered the arithmetic check incorrectly (3463 instead of 3563) and was rejected.

This MSE artifact passed that arithmetic check and the complete serving gate, so it was the only artifact promoted.

Reproduce the validated DGX Spark serving path

Download this target and the external draft:

hf download DevelopingDad/Qwen3.8-27B-Uncensored-NVFP4-MSE-DGX-Spark \
  --local-dir /path/to/target

hf download maurienne-ai/Qwen3.8-27B-DFlash2-NVFP4-RTNcal \
  --revision bd7a934213c47a9e7ef69eef36bb3325f47fd1f1 \
  --local-dir /path/to/draft

Launch the exact validated SGLang image:

docker run --rm --gpus all --network host --ipc host --shm-size 32g \
  -e HF_HUB_OFFLINE=1 \
  -e TRANSFORMERS_OFFLINE=1 \
  -e SGLANG_OPT_MAMBA_SKIP_DECODE_LOCK=0 \
  -v /path/to/target:/models/target:ro \
  -v /path/to/draft:/models/draft:ro \
  lmsysorg/sglang@sha256:00205b89f74691f76a0ffbd6846376d9323971930a5d59bf63a65dadc7d67927 \
  python3 -m sglang.launch_server \
    --model-path /models/target \
    --served-model-name Qwen3.8-27B-Uncensored-NVFP4-MSE \
    --host 127.0.0.1 \
    --port 30138 \
    --mem-fraction-static 0.80 \
    --sleep-on-idle \
    --attention-backend flashinfer \
    --fp8-gemm-backend cutlass \
    --chunked-prefill-size 8192 \
    --disable-prefill-cuda-graph \
    --kv-cache-dtype bfloat16 \
    --mamba-ssm-dtype bfloat16 \
    --mamba-full-memory-ratio 4.21 \
    --mamba-radix-cache-strategy extra_buffer \
    --max-mamba-cache-size 20 \
    --max-running-requests 4 \
    --context-length 262144 \
    --speculative-algorithm DFLASH \
    --speculative-draft-model-path /models/draft \
    --speculative-draft-model-quantization modelopt_fp4 \
    --speculative-num-draft-tokens 8 \
    --reasoning-parser qwen3 \
    --tool-call-parser qwen3_coder \
    --sampling-defaults model \
    --enable-metrics \
    --enable-cache-report

See `SERVING.md` for the validation checklist and rollback notes.

Validated single-Spark result

Hardware: one NVIDIA DGX Spark / GB10 with 128 GB unified memory.

Primary workload: concurrency 1, temperature 0, thinking disabled, fixed 128 output tokens, one discarded warm-up followed by seven measured requests.

Median time to first token improved from 0.2984 s to 0.1923 s (-35.6%). A five-content suite (programming list, Python code, systems prose, worked math, and French prose) improved from a pooled median of 24.062 to 34.008 tok/s (+41.3%). Every content-class median improved by more than 25%.

This is a system-level DGX Spark result, not an intrinsic speed guarantee for the weight files. The baseline used the FP8 target plus an unquantized DFlash2 draft; the winning system uses this mixed target plus the calibrated NVFP4 DFlash2 draft. It is not a pure target-weight ablation.

Machine-readable speed and capability details are in `benchmark-results.json`.

Capability results

The promoted configuration was also evaluated on the same four benchmark families and sample counts reported by the OrcaRouter parent card. All 1,200 requests ran on one DGX Spark with temperature 0, thinking disabled, a fixed sample seed, and concurrency 4 (the service's configured low-concurrency ceiling).

The endpoint retained the legacy served-model alias orcarouter/Qwen3.8-27B-Uncensored-FP8. Container-mount inspection and fresh SHA-256 checks of all three target shards confirmed that /models/target was this published C-MSE artifact; the API alias was not used as the model-identity proof.

The Qwen-base and OrcaRouter-parent columns are quoted from the `orcarouter/Qwen3.8-27B-Uncensored` capability table. That card does not publish its sample IDs, random seed, exact prompts, generation limits, or scorer. This model's column is therefore a new pinned run of the same benchmark families and sample counts, not a matched rerun of the hidden upstream protocol. The deltas are descriptive and should not be read as proof that quantization or SGLang improved the model's intrinsic capability.

The run used the pinned SGLang/OpenAI Simple Evals generative MMLU prompt, five category-matched CoT examples for MMLU-Pro, the community-standard eight-example GSM8K CoT prompt, and zero-shot Chinese letter selection for CMMLU. The GSM8K prompt difference is especially important: the parent card only says CoT, so the large GSM8K delta must not be attributed to the checkpoint or runtime.

Every response was retained and failures were scored as failures. Six MMLU responses and 30 MMLU-Pro responses did not produce an extractable final answer; all were counted incorrect. GSM8K and CMMLU had 100% answer extraction. The complete run had zero request errors. Dataset revisions, sample seed, answer-extraction rules, parse counts, and per-category results are recorded in `benchmark-results.json`.

Serving validation

The promoted configuration passed the full gate before restart, after a cold restart, and on the persistent production port:

• —exact sentinel response
• —arithmetic correctness
• —strict JSON schema output
• —automatic tool call and tool-result round trip
• —reasoning/content separation
• —image color identification
• —context recall
• —incremental streaming with exactly 128 output tokens
• —cancellation followed by a clean new request

Additional exact retrieval checks passed at 120,003 and 199,999 prompt tokens. These are targeted serving checks, not a comprehensive benchmark, safety evaluation, or proof of full 262K reasoning parity.

Compatibility notes

• —The validated performance path is SGLang with ModelOpt mixed-weight support.
• —A generic Transformers load does not reproduce the reported SGLang/DFlash performance.
• —Runtime support for ModelOpt packed layers is required; not every Transformers backend can execute this checkpoint directly.
• —The external draft checkpoint remains subject to its own repository terms and compatibility requirements.
• —Results are from one DGX Spark and a single-stream/low-concurrency workload. Other hardware, prompts, sampling settings, or concurrency levels may behave differently.

Credits, attribution, and modification notice

This repository contains transformed weight and configuration files. DevelopingDad produced the ModelOpt mixed-precision export and the accompanying DGX Spark validation materials; the parent model training and abliteration were performed upstream.

• —Qwen Team / Alibaba Cloud created `Qwen/Qwen3.8-27B`.
• —OrcaRouter / Continuum AI created the abliterated parent `orcarouter/Qwen3.8-27B-Uncensored`.
• —NVIDIA created Model Optimizer, used to produce this C-MSE mixed NVFP4/FP8 export.
• —maurienne.ai published the separately distributed calibrated `Qwen3.8-27B-DFlash2-NVFP4-RTNcal` draft used by the reported serving configuration.
• —That draft derives from Inco AI's `incoai/Qwen3.8-27B-DFlash2` and the DFlash research by Jian Chen, Yesheng Liang, and Zhijian Liu.
• —The validated runtime uses the community SGLang project.

This independent community derivative is not affiliated with, endorsed by, or sponsored by Alibaba Cloud/Qwen, OrcaRouter/Continuum AI, NVIDIA, maurienne.ai, Inco AI, SGLang, or the DFlash authors. Their names and trademarks are used only for identification and attribution.

See `ATTRIBUTION.md` for consolidated provenance, license, and external-component notes.

Provenance summary

• —Qwen base: `Qwen/Qwen3.8-27B`
• —Abliterated parent: `orcarouter/Qwen3.8-27B-Uncensored`
• —Parent revision: 9878936be9458522b5aeed0e13476bb8426f57f0
• —Quantization: NVIDIA ModelOpt C-MSE mixed NVFP4/FP8 PTQ
• —Draft used for the reported result: `maurienne-ai/Qwen3.8-27B-DFlash2-NVFP4-RTNcal` at bd7a934213c47a9e7ef69eef36bb3325f47fd1f1
• —Validation date: 2026-09-22

Citation

If you use this weight artifact, please cite the artifact and the upstream Qwen model. If you use the reported speculative-serving configuration, please also cite DFlash 2, the original DFlash work, and NVIDIA ModelOpt.

@misc{developingdad2026qwen38nvfp4mse,
  title     = {{Qwen3.8-27B-Uncensored-NVFP4-MSE-DGX-Spark}},
  author    = {{DevelopingDad}},
  year      = {2026},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/DevelopingDad/Qwen3.8-27B-Uncensored-NVFP4-MSE-DGX-Spark},
  note      = {ModelOpt C-MSE mixed NVFP4/FP8 derivative with a validated SGLang DFlash2 recipe}
}

@misc{qwen38,
  title  = {{Qwen3.8-Max}: A New Bar for Coding and Cowork},
  url    = {https://qwen.ai/blog?id=qwen3.8},
  author = {{Qwen Team}},
  month  = {August},
  year   = {2026}
}

@misc{inco2026dflash2,
  title  = {{DFlash 2: Keep Drafting Parallel}},
  author = {{Inco AI}},
  year   = {2026},
  month  = {August},
  url    = {https://inco.ai/blog/dflash2/}
}

@inproceedings{chen2026dflash,
  title     = {{DFlash: Block Diffusion for Flash Speculative Decoding}},
  author    = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
  booktitle = {International Conference on Machine Learning (ICML)},
  year      = {2026}
}

@misc{nvidia-modelopt,
  author       = {{NVIDIA Corporation}},
  title        = {{NVIDIA Model Optimizer}},
  howpublished = {\url{https://github.com/NVIDIA/Model-Optimizer}},
  year         = {2024--2026},
  note         = {GitHub repository}
}

License

Apache 2.0. The included LICENSE is byte-for-byte identical to the license distributed by both the Qwen base and the pinned OrcaRouter parent. Neither upstream repository included a NOTICE file at the audited revisions. Retain upstream attribution and comply with the terms of any separately downloaded draft model.