Wondernutts/Qwen3.8-27B-Uncensored-int4-awq-g128-ov

OpenVINO INT4 AWQ — asymmetric 4-bit weight compression, verified group size 128.

Qwen3.8 27B Uncensored — OpenVINO INT4 AWQ g128

OpenVINO INT4 AWQ deployment conversion of JonathanColetti/Qwen3.8-27B-Uncensored, based on Qwen/Qwen3.8-27B.

Jonathan Coletti performed the original refusal-direction removal and published the BF16 source checkpoint. Qwen created the base model. Wondernutts performed this OpenVINO conversion and controlled deployment validation; this repository does not claim authorship of the upstream model work.

The original plain-INT4 OpenVINO baseline remains untouched at Wondernutts/Qwen3.8-27B-Uncensored-int4-ov.

Verified conversion facts

The uploaded openvino_config.json and conversion manifest record:

• —Weight format: asymmetric INT4
• —Compression method: AWQ
• —Group size: 128
• —Dynamic-quantization group size: 128
• —Compression ratio: 1.0
• —OpenVINO/Optimum export metadata: Optimum 2.3.0, Transformers 5.2.0
• —Source revision: JonathanColetti/Qwen3.8-27B-Uncensored@5bb7aa90f0efef548e87005b1fb7658e522b6b7f
• —Base revision: Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0
• —Source tensor inventory: 1,199 keys, including all 15 verified mtp.* source tensors
• —Complete multimodal OpenVINO deployment artifacts: language model, tokenizer/detokenizer, text embeddings, and vision embedding/merger/position graphs

MTP caveat

The BF16 source checkpoint contains all 15 verified MTP tensors, restored from the Qwen base model and checked before conversion. However, the standard OpenVINO VLM export used here does not expose a separate MTP draft head. Do not interpret the source-tensor verification as a claim that this OpenVINO package performs MTP speculative decoding.

Controlled A/B validation

The repository includes AB_VALIDATED.json, comparing this AWQ candidate against the untouched plain-INT4 Wondernutts baseline. The record reports passed: true; both models produced the expected math answer, coherent roleplay output, and correct red-image recognition.

This controlled CPU DYNAMIC_QUANTIZATION_GROUP_SIZE=0 smoke A/B shows no meaningful generation-latency regression in these three checks. It is a functional comparison, not a throughput benchmark: no tokens-per-second claim is made, and the small timing differences should not be generalized beyond this run and configuration.

The validation file also records that the candidate is AWQ while the preserved baseline is not; both use group-128 INT4 storage.

One-B70 Intel Arc benchmark

Measured with the established qwen38_arc_bench.py harness on one Intel Arc Pro B70, OpenVINO runtime 5c2085a1f7a8, with thinking disabled. These are results from this specific system and harness, not universal performance claims.

Recommended serving profile: DQ128

DQ0 control

Both profiles passed the math and Skyrim roleplay-coherence gates. DQ128 is the recommended serving profile: decode performance is effectively tied, while 12K warm prompt processing is approximately 33.7% faster than DQ0 (2,216.57 versus 1,657.47 tok/s). The earlier CPU DQ0 smoke A/B above remains a functional candidate-versus-baseline check; it should not be conflated with this Arc GPU benchmark.

Source-model context

The source model substantially reduces—but does not eliminate—refusal behavior from Qwen3.8-27B using Heretic. Its source card reports the original methodology, measured behavior, limitations, and safety caveats. Review both the uncensored source card and Qwen base card before deployment.

Wondernutts OpenVINO work

• —Gemma/Qwen conversion, validation, and serving tooling: Wondernuttz/OpenVino-For-Gemma-4
• —Custom OpenVINO runtime research: Wondernuttz/openvino

These links provide conversion/runtime provenance. They are not claims of authorship over the Qwen or JonathanColetti source weights.

Inference with OpenVINO GenAI

~~~python import openvinogenai as ovgenai from huggingfacehub import snapshotdownload

modeldir = snapshotdownload( "Wondernutts/Qwen3.8-27B-Uncensored-int4-awq-g128-ov" )

pipe = ovgenai.VLMPipeline( modeldir, "GPU", DYNAMICQUANTIZATIONGROUP_SIZE=128, )

config = ovgenai.GenerationConfig() config.maxnewtokens = 512 config.dosample = True config.temperature = 1.0 config.topp = 0.95 config.topk = 20

result = pipe.generate("Write a short scene in a candlelit inn.", generation_config=config) print(result) ~~~

For direct answers, apply the repository's chat template with thinking disabled where supported. Native image input requires the multimodal VLMPipeline API and a compatible OpenVINO GenAI build.

Notes

• —This is an OpenVINO deployment repository, not a Transformers checkpoint. Use OpenVINO/OpenVINO GenAI rather than AutoModelForImageTextToText.
• —Reduced-refusal behavior is not the same as unrestricted correctness. The source model retains documented refusal and capability limitations.
• —License: Apache-2.0, inherited from the source/base model. Review upstream terms before redistribution or commercial deployment.