khazarai/Qwen3-4B-Gemini-3.1-Pro-Reasoning-Distilled

Qwen3-4B-Gemini-3.1-Pro-Reasoning-Distilled : GGUF

Qwen3-4B-Gemini-3.1-Pro-Reasoning-Distilled is a reasoning-enhanced language model fine-tuned from the Qwen3-4B-Thinking-2507 base model using a high-density synthetic reasoning dataset distilled from Gemini 3.1 Pro.

The goal of this fine-tuning is to improve multi-step reasoning, structured problem decomposition, and analytical thinking while maintaining the strong language capabilities of the base Qwen model.

This model focuses on System-2 style reasoning, emphasizing logical derivations, verification steps, and structured explanations.

• —Model name: khazarai/Qwen3-4B-Gemini-3.1-Pro-Reasoning-Distilled
• —Base model: Qwen3-4B-Thinking-2507

Available Model files:

• —qwen3-4b-thinking-2507.BF16.gguf
• —qwen3-4b-thinking-2507.Q8_0.gguf
• —qwen3-4b-thinking-2507.Q4_K_M.gguf

These files enable deployment in local inference frameworks such as llama.cpp or Ollama.

Training Method

Method:

• —QLoRA (4-bit quantization)
• —LoRA adapters
• —Training framework: Unsloth

Dataset

The model was trained on a synthetic reasoning dataset distilled from Gemini 3.1 Pro.

Dataset Overview

Total Token Volume: 5.6 million tokens The dataset prioritizes logical density and deep reasoning traces, rather than large token counts. The dataset represents a frontier approach in reasoning distillation, where a stronger reasoning model generates structured problem-solving examples that are then used to train a smaller model.

Dataset generation pipeline:

• —Question generation
• —Multi-step reasoning derivation
• —Answer verification

Curated using an agentic workflow involving:

• —Gemini 3 Flash
• —Gemini 3.1 Pro

Domain Coverage

The dataset spans 60+ expert-level domains, focusing on complex reasoning tasks.

• —Physics
• —Mathematics
• —Computer Science
• —Biology & Medicine
• —Strategic Reasoning
• —Advanced Benchmarks

The dataset includes problems inspired by challenging benchmarks such as:

• —ARC-AGI
• —GPQA Diamond
• —MMLU-Pro
• —HLE Benchmark

Intended Use

This model is designed for:

• —Complex reasoning tasks
• —Scientific explanations
• —Advanced mathematics and logic
• —Programming and computer science questions
• —Analytical problem solving

Typical use cases include:

• —research assistance
• —algorithm reasoning
• —technical education
• —complex question answering

Limitations

• —Despite improved reasoning ability, the model still inherits limitations from its base architecture:
• —Hallucinations may still occur in specialized domains.
• —Synthetic reasoning datasets may introduce stylistic biases.
• —Performance may degrade on tasks requiring extremely long context or domain-specific expertise.

Users should verify outputs when used in critical scientific or technical environments.