Mochgolf/Qwen3.8-Flash-Next-Uncensored-W4A16-AutoRound-INT8PLE-MTP
Qwen3.8-Flash-Next-Uncensored — AutoRound W4A16 / INT8 PLE
This is a locally assembled and tested quantization of `orcarouter/Qwen3.8-Flash-Next-Uncensored`, which derives from `Qwen/Qwen3.8-Flash-Next`. The language-model routed experts were quantized directly from the OrcaRouter BF16 checkpoint with standard AutoRound; this is not a refusal edit applied to an already-quantized Intel checkpoint. The model also carries an independently stored INT8 row-wise PLE n-gram embedding and retains the vision and MTP paths.
Safety: The source model has had refusal behavior deliberately removed. It has no reliable built-in safety guardrails. Use only for lawful, controlled research or applications with appropriate external moderation and abuse prevention. Do not expose it to end users without safeguards.
Verified serving stack: [QSA HiSparse](https://github.com/mochgolf/qsa-hisparse). This maintained, Apache-2.0 SGLang source fork implements Qwen Sparse Attention with CPU KV offload and supports the INT8 row-wise PLE offload used here. The project's runtime guide, setup instructions, and service controller are the starting points for reproducing this deployment. The measurements below used the pinned source commit, not every configuration or future revision of the fork.
Provenance and quantization
The independent PLE sidecar is a deliberate mixed-provenance serving choice. The OrcaRouter checkpoint's modified PLE projection weights are retained; the large n-gram embedding is supplied by the row-wise INT8 sidecar. Selected source PLE tensors were checked during assembly, but this card does not claim an exhaustive BF16-vs-INT8 numerical equivalence proof for the table. The assembly used copies rather than hard links, so this release does not depend on the local donor files.
Related quantizations
`bidhata/Qwen3.8-Flash-Next-AutoRound-Uncensored` is a close relative, but its card describes transferring an uncensoring edit onto a pre-quantized Intel/Saren AutoRound hybrid and using an FP8 PLE table. `Saren/Qwen3.8-Flash-Next-W4A16-AutoRound-hybrid` is an AutoRound-based serving hybrid of the standard, non-uncensored Qwen model. This release differs in the direct OrcaRouter-BF16 quantization path and its INT8 row-wise PLE. It is not presented as the first or only Uncensored/AutoRound quantization.
Evaluation
The full frozen EvalPlus code-generation run used one greedy completion per task, max_tokens=1536, thinking disabled, and EvalPlus 0.3.1. Generated code was scored only in an offline, unprivileged container. All 542 API requests completed; two outputs reached the generation limit. The serial scorer produced 542 valid rows and zero internal scorer errors.
The stricter plus score is the primary public code-generation result. Scores on these established datasets can be affected by training-data overlap, so they are not proof of generalization. Detailed aggregate counts and the frozen protocol are in benchmark/evalplus.json and BENCHMARK.md.
The DSH coding-agent run passed 24/24 project-owned validation tasks (0 failures, 0 errors), using 150 local model calls and 131 tool calls. Median end-to-end task time was 22.246 seconds; the sum of task times was 589.304 seconds. This measures a small, non-blind tool-using workflow rather than a public general-agent score. Task texts, workspaces and transcripts are not redistributed here; the recovery gap in the local aggregator is excluded from the reported task times. See BENCHMARK.md and the aggregate JSON.
Earlier serving checks on the same local checkpoint and qsa-hisparse@ee8fe158d64186b47236b007a299696030c372e8 completed 704/704 requests in a 10-minute B8 soak with zero observed request errors. A separate fixed long-decode test measured 80.493 output tokens/s for one request after warmup. These figures describe specific local serving workloads, not an architecture-wide speed claim or a controlled A/B against the BF16 source.
Serving
The verified runtime was the QSA HiSparse SGLang fork at commit `ee8fe158`, tensor-parallel on two GPUs with 48 GiB memory each and CPU-backed PLE offload. Other SGLang, Transformers, vLLM and llama.cpp loading paths are not verified for this mixed format. The essential runtime options were:
python -m sglang.launch_server \
--model-path /path/to/model \
--served-model-name qwen3.8-flash-next-uncensored-w4a16-int8ple \
--model-impl sglang --tp-size 2 \
--dtype bfloat16 --ple-offload-embedding \
--json-model-override-args '{"text_config":{"ple_embedding_dtype":"int8_row"}}' \
--kv-cache-dtype fp8_e4m3 --linear-attn-backend triton \
--reasoning-parser qwen3 --tool-call-parser qwen3_coderThe measured deployment additionally set cache, memory and CUDA-graph options; the exact relevant settings are documented in BENCHMARK.md. The short command above is a loading example, not the measured full profile. The INT8 PLE sidecar is required; do not omit model-ple-rowint8.safetensors.
License and limitations
The actual LICENSE distributed with the official Qwen base and this model is the Qwen Community License 1.0. It permits publication of derivatives subject to its conditions, including preserving its notice. In particular, commercial Model-as-a-Service or AI Work Assistant use by a licensee may require a separate Qwen license; read the full included LICENSE before using the model. The OrcaRouter source model card currently labels itself Apache-2.0, which conflicts with the official Qwen base repository and its license file. This release does not present the Qwen-origin weights as Apache-only.
Only a narrow set of coding, agent and serving workloads has been tested here. The BF16 source was not run locally as a matched benchmark arm, so differences cannot be attributed solely to quantization. Long-context, multimodal, security and broad safety behavior have not been independently benchmarked for this release.