Modelpublic

Alittlehammmer/gemma-4-31B-it-DFlash-GGUF-llama.cpp

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face

2likes462downloads

Model Card

Gemma-4-31B-it-DFlash

GGUF quantizations of z-lab/gemma-4-31B-it-DFlash.

Converted to BF16 using convert_hf_to_gguf.py, then quantized using llama-quantize from llama.cpp.

Available quants

Quant	Bits	Size	Notes
Q4KM	4	912 MB	Average quality
Q5_K	5	1.09 GB	High quality
Q6_K	6	1.27 GB	Very high quality
Q8_0	8	1.65 GB	Highest quality, near lossless, Recommended
BF16	16	3.07 GB	Full precision, reference file

Usage

Use in conjunction with existing Gemma 4 Quants, example config if using llama-server:

bash

[Gemma-4-31B-it-DFlash]
sm = layer
model = /mnt/gguf/Gemma-4-31B-it/Gemma-4-31B-it-Q8_0.gguf
model-draft = /mnt/gguf/Gemma-4-31B-it-DFlash/Gemma-4-31B-it-DFlash-Q8_0.gguf
spec-type = draft-dflash
spec-draft-n-max = 6

(Note: For some reason I cannot get sm = tensor to work, it crashes on launch, pretty sure this is an issue in llama.cpp)

Original model

See the original model card for details on capabilities, benchmarks, and license.