fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090

Qwen3.8-27B Uncensored NVFP4 RTX 5090

RTX 5090 optimized ModelOpt NVFP4 conversion of orcarouter/Qwen3.8-27B-Uncensored, produced directly from the original BF16 safetensors.

This checkpoint preserves the Qwen3.8 vision-language architecture and native MTP tensors. It is paired with the target-matched fleetml/Qwen3.8-27B-Uncensored-DSpark-RTX5090, now released in ModelOpt NVFP4.

Combined release

The validated Hermes serving profile launches with a 122,880-token context limit. SGLang automatically allocated an 87,798-token active pool and left 2.83 GB of GPU memory free after cache allocation and CUDA graph capture on one RTX 5090.

Measured performance

The current NVFP4 target and NVFP4 drafter profile reached 217.73 tokens per second median decode across 18 successful requests on one RTX 5090.

Test conditions were concurrency 1, temperature 0, two warmups, three repeats across six prompts, FP8 E4M3 KV cache, and a fixed request seed. Decode throughput excludes prompt processing and time to first token. The 16,000-token request setting was a ceiling. This result is not a sustained 16,000-token generation measurement.

The earlier deterministic BF16 drafter profile measured 139.32 tokens per second with a 16,384-token active pool. Its full standard decoding, native MTP, public DSpark, and trained DSpark comparison remains available in benchmarks/benchmark-summary.json.

Checkpoint

Run with the matched NVFP4 DSpark drafter

docker run --rm --gpus all --ipc=host -p 30000:30000 \
  lmsysorg/sglang:qwen38-27b \
  sglang serve \
  --model-path fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090 \
  --quantization modelopt_fp4 \
  --trust-remote-code \
  --speculative-algorithm DSPARK \
  --speculative-draft-model-path fleetml/Qwen3.8-27B-Uncensored-DSpark-RTX5090 \
  --speculative-draft-model-quantization modelopt_fp4 \
  --speculative-dspark-block-size 7 \
  --speculative-num-steps 1 \
  --speculative-eagle-topk 1 \
  --context-length 122880 \
  --max-mamba-cache-size 8 \
  --kv-cache-dtype fp8_e4m3 \
  --attention-backend flashinfer \
  --chunked-prefill-size 1024 \
  --mamba-radix-cache-strategy extra_buffer_lazy \
  --mamba-ssm-dtype bfloat16 \
  --mm-feature-transport cpu \
  --cuda-graph-max-bs-decode 1 \
  --mem-fraction-static 0.86 \
  --max-running-requests 1 \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder \
  --host 0.0.0.0 \
  --port 30000

The active token pool is allocated automatically. Available capacity can vary slightly with the runtime build and GPU state.

Provenance

• —Base model: Qwen/Qwen3.8-27B
• —BF16 source: orcarouter/Qwen3.8-27B-Uncensored
• —Quantization: NVIDIA ModelOpt 0.47.0.dev70, commit c4129b6e
• —Calibration: 126 tokenized UltraChat conversations
• —Validation: complete shard, tensor, dtype, MTP, load, API, Hermes, and benchmark checks

Use

This checkpoint is intended for controlled local research, evaluation, and agent development. Use it responsibly and comply with the Apache 2.0 license and applicable law.