cloudbjorn/gemma-4-31B-Opus-4.6-Reasoning-GGUF

Gemma-4-31B-Opus-4.6-Reasoning-GGUF

This is a quantized GGUF version of the fine-tuned and merged Gemma-4-31B_Opus-4.6-Reasoning-3300x model.

Introduction

The primary goal of this project was to leverage Gemma-4's native <|channel> architecture to enforce strict, logical step-by-step reasoning. By fusing the Gemma-4 foundation with the Opus reasoning dataset, the model acts as a deeply analytical agent.

This repository provides ultra-efficient .gguf archives. These files enable you to run exactly this reasoning model entirely on consumer-grade hardware (like a MacBook or modest GPU setup) without requiring large amounts of VRAM.

Benchmarks & Performance

While this is a 4-bit quantization, the model retains the logical density of the original BF16 merge.

• —Original BF16 ARC-Challenge (acc_norm): 69.54%
• —Original Base (Gemma-4-31B) ARC-Challenge (acc_norm): 69.88%

The structural output improvements remain consistent across quants, maintaining strict <|channel>thought integrity.

Reasoning Format: Gemma 4 Architecture

This model adheres strictly to the Gemma 4 multimodal and reasoning formats. It outputs internal reasoning within <|channel>thought bounds before delivering the final response.

Below are raw, unedited samples from this quant:

User: I need to fine-tune a 70B parameter model. I have exactly two 80GB A100 GPUs. Walk me through the math of whether I can do a full-parameter fine-tune in pure 16-bit, or if I am mathematically forced to use 4-bit quantization and LoRA. Show your exact VRAM calculations. Model: <|channel>thought Let me work through this step by step: 1. Model Size Calculation - Model size = 70B parameters 2 bytes (fp16) = 140,000,000,000 bytes - Convert to GB: 140,000,000,000 / 1,024^3 ≈ 128.19 GB ... 6. Memory Comparison - Required: 512.76 GB - Available: 160 GB - Shortfall: 512.76 - 160 = 352.76 GB ### Conclusion With two 80GB A100s, you cannot perform a full-parameter fine-tune in pure 16-bit precision due to insufficient VRAM. `<channel|>` Final Answer:* You will need to use either: 1. 4-bit quantization (QLoRA) with LoRA adapters 2. ZeRO-3 offloading ...

Training Details (from original BF16 model)

• —Base Model: google/gemma-4-31B
• —Dataset: Crownelius/Opus-4.6-Reasoning-3300x
• —Training Framework: Eschaton Engine (Cloudbjorn)
• —Format: Merged (Base + LoRA)

Training Precision:

• —Compute Dtype: bfloat16

LoRA Parameters (Auto-Scaled for 31B via Eschaton):

• —r: 16
• —lora_alpha: 32
• —target_modules: all-linear

Hyperparameters:

• —Optimizer: 8-bit Paged AdamW
• —Effective Batch Size: 32 (Gradient Accumulation)
• —Learning Rate: 2e-5
• —LR Scheduler: Linear
• —Epochs: 1
• —Training Sequence Length: 2048
• —Warmup Steps: 50
• —Weight Decay: 0.01