Ayodele01/Gemma-4-12B-Gemini-3.5-flash-Reasoning-Distill-GGUF

Gemma-4-12B-Gemini-3.5-flash-Reasoning-Distill-GGUF

GGUF quantized versions of Ayodele01/Gemma-4-12B-Gemini-3.5-flash-Reasoning-Distill.

Model Description

This is Google's Gemma-4 12B instruction-tuned model, fine-tuned on the full 25,000 synthetic reasoning examples dataset WithinUsAI/gemini_3.5_flash_distilled_25k using QLoRA via Unsloth.

This GGUF model contains quantized versions of the merged model weights.

Available Files and Quantizations

Usage with llama.cpp

You can run these files using llama.cpp.

# Run with llama-cli
./llama-cli -m Gemma-4-12B-Gemini-3.5-flash-Reasoning-Distill-Q5_K_M.gguf \
  -p "<|turn>user\nWhat is the sum of all prime numbers between 1 and 50?<|turn>model\n" \
  -n 512

Prompt Template

Gemma-4 chat template format:

<|turn>user
{ prompt }<|turn>model

Training and Distillation Context

For details on evaluations, training hyperparameters, and qualitative findings, please refer to the main repository model card: Ayodele01/Gemma-4-12B-Gemini-3.5-flash-Reasoning-Distill.