leon-se/gemma-3-27b-it-FP8-Dynamic
208.9k
FP8-Dynamic quantization using llmcompressor. Run with:
vllm serve leon-se/gemma-3-27b-it-FP8-Dynamic --max-model-len 4096FP8-Dynamic quantization using llmcompressor. Run with:
vllm serve leon-se/gemma-3-27b-it-FP8-Dynamic --max-model-len 4096