Modelpublic

rhoninseiei/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-ModelOpt-NVFP4

sourceHugging Faceapache-2.0updated 6mo agoView on Hugging Face

1likes14downloads

Model Card

Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-ModelOpt-NVFP4

This repository contains a ModelOpt-quantized checkpoint derived from `Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled`.

Quantization Summary

—Quantization tool: NVIDIA TensorRT Model Optimizer
—Weight quantization: NVFP4
—KV cache quantization: FP8
—Calibration size: 1024 samples
—Calibration sequence length: 4096
—Calibration batch size: 1
—Calibration source mix:
—nohurry/Opus-4.6-Reasoning-3000x-filtered: 596 samples
—Jackrong/Qwen3.5-reasoning-700x: 178 samples
—TeichAI/claude-4.5-opus-high-reasoning-250x: 250 samples

The calibration set was converted into a single JSONL file with the source model's chat template applied before PTQ, so the activation distribution is closer to the reasoning format used by this distilled checkpoint.

Runtime Notes

—Intended runtime target: SGLang with ModelOpt-compatible HF checkpoint loading
—Quantization format: ModelOpt HF export, not compressed-tensors
—A few unsupported or intentionally skipped modules may remain excluded by ModelOpt during export; see hf_quant_config.json for the final exclusion list

Files

—model.safetensors: quantized weights
—hf_quant_config.json: final quantization metadata
—tokenizer and processor files inherited from the source checkpoint