jamesbrunet/Swift-Qwen3.8-27b-W4A16-AutoRound

Swift-Qwen3.8-27b-W4A16-AutoRound

Quantized ukisai/Swift-Qwen3.8-27b with Intel AutoRound and tested in vLLM. It appears to work!

Recipe

Copied the work of dbirks/Qwen3.8-27B-W4A16-AutoRound except with embedding unquantized.

License

Base model distributed under UkisAI's Swift Open License v1.0. This quantization probably inherits those terms, but I'm not a lawyer.