maximg/AutoRoundTest

Qwen3.6-27B GGUF (AutoRound Quantized, MTP Enabled)

This repository contains GGUF quantized versions of Qwen/Qwen3.6-27B created using Intel's AutoRound quantization method.

Quantization Details

The models were generated using Intel's AutoRound using ultrachat_200k as the test dataset and using sequence length of 2850. MTP layers were not explicitly enabled, but it works with MTP for me

auto-round \
    --model Qwen/Qwen3.6-27B \
    --output_dir ./quantized/ \
    --scheme <SCHEME> \
    --format <SCHEME> \
    --iters 0 \
    --nsamples 256 --seqlen 2850 --dataset "HuggingFaceH4/ultrachat_200k"

For now, only 2 quantization variants were used Q5KM and Q4KMIXED. Q4KMIXED is a custom variant based on Intel's original Q2KMIXED quantization, but using Q4_K quants instead of Q2.

Files and Sizes