rico03/Qwen3.6-27B-Claude-Opus-Reasoning-Distilled

Qwen3.6-27B — Claude Opus Reasoning Distilled

<p align="center"> <img src="https://img.shields.io/badge/Base%20Model-Qwen3.6--27B-blue?style=for-the-badge"/> <img src="https://img.shields.io/badge/Training-LoRA%20r64%20%7C%20Unsloth-orange?style=for-the-badge"/> <img src="https://img.shields.io/badge/Distilled%20From-Claude%204.6%20Opus-purple?style=for-the-badge"/> <img src="https://img.shields.io/badge/License-Apache%202.0-green?style=for-the-badge"/> </p>

Qwen3.6-27B fine-tuned with ~14k Claude 4.6 Opus reasoning traces — structured, efficient thinking for coding, math, and analytical tasks.

🙏 This model was trained following the methodology and pipeline guide by Jackrong, adapted for Qwen3.6-27B and extended with additional datasets and quantization options.

📦 Looking for GGUF quantized versions (llama.cpp, Ollama)? → rico03/Qwen3.6-27B-Claude-Opus-Reasoning-Distilled-GGUF

🎯 Why This Model Exists

Qwen3.6-27B is one of the most capable open-weight 27B models ever released — it outperforms models 10× its size on coding benchmarks and rivals closed frontier models. But raw capability alone isn't enough.

The base model has a known weakness: verbose, repetitive reasoning loops on straightforward queries. It over-thinks simple tasks and produces unnecessarily long chains of thought that hurt inference speed and readability.

This fine-tune addresses that directly by distilling the structured, efficient reasoning style of Claude 4.6 Opus into Qwen3.6-27B. The goal is not to change what the model knows, but how it thinks:

• —✅ Structured <think>...</think> before every response
• —✅ Concise reasoning on simple tasks, deep analysis on hard ones
• —✅ Claude-style step-by-step decomposition
• —✅ Reduced redundant cognitive loops
• —✅ Preserved base model capabilities

🧠 Learned Reasoning Pattern

The model adopts a Claude-style structured reasoning scaffold:

<think>
Let me analyze this request carefully:

1. Identify the core objective of the problem.
2. Break the task into clearly defined subcomponents.
3. Evaluate constraints and edge cases.
4. Formulate a step-by-step solution plan.
5. Execute the reasoning sequentially and verify consistency.
</think>

[Final Answer]

📊 Base Model Benchmarks (Qwen3.6-27B)

Qwen3.6-27B is the base model. These are its official benchmark results — the fine-tune inherits this capability while improving reasoning structure.

[image]

Language & Coding

Knowledge & Reasoning

Source: [Qwen3.6-27B official release](https://qwen.ai/blog?id=qwen3.6-27b)

Fine-tuned Model

🗺️ Training Pipeline

Base Model: Qwen/Qwen3.6-27B (27B dense, multimodal)
        │
        ▼
4-bit quantized loading via Unsloth
        │
        ▼
LoRA Rank-64 Adapter attached
(q_proj, k_proj, v_proj, o_proj,
 gate_proj, up_proj, down_proj, out_proj)
        │
        ▼
SFT — Response-Only Training
Masked on: "<|im_start|>assistant\n<think>"
Chat template: qwen3-thinking
        │
        ▼
rico03/Qwen3.6-27B-Claude-Opus-Reasoning-Distilled

📚 Datasets

Total: ~14,233 examples after normalization, deduplication and length filtering (max 8192 tokens).

⚙️ Training Configuration

💻 Usage

Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "rico03/Qwen3.6-27B-Claude-Opus-Reasoning-Distilled"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [{"role": "user", "content": "Implement a binary search tree in Python with insert and search methods."}]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=4096,
    temperature=0.6,
    top_p=0.95,
    top_k=20,
)
print(tokenizer.decode(outputs[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True))

vLLM

pip install vllm

vllm serve rico03/Qwen3.6-27B-Claude-Opus-Reasoning-Distilled \
  --port 8000 \
  --max-model-len 8192 \
  --reasoning-parser qwen3

⚡ Speculative Decoding with MTP (vLLM)

Qwen3.6 supports Multi-Token Prediction (MTP) for significantly faster inference. Community tests show ~90% acceptance rate on this fine-tuned model — higher than typical, thanks to the structured reasoning training. Generation throughput reaches 60+ tok/s with MTP enabled, compared to ~25 tok/s standard.

vllm serve rico03/Qwen3.6-27B-Claude-Opus-Reasoning-Distilled \
  --port 8000 \
  --max-model-len 8192 \
  --reasoning-parser qwen3 \
  --speculative-config '{"method":"qwen3_next_mtp","num_speculative_tokens":2}'

SGLang

python -m sglang.launch_server \
  --model-path rico03/Qwen3.6-27B-Claude-Opus-Reasoning-Distilled \
  --port 8000 \
  --reasoning-parser qwen3

Recommended Sampling Parameters

⚠️ Limitations

• —Text-only SFT: vision capabilities of the base model are not fine-tuned
• —1 epoch: trained for 1 epoch on ~14k samples
• —Hallucination risk: autoregressive LLM — may produce incorrect facts
• —Intended use: coding, math, offline analytical tasks, logic-heavy prompting

📖 Citation

@misc{rico03-qwen36-opus-reasoning,
  title  = {Qwen3.6-27B Claude Opus Reasoning Distilled},
  author = {rico03},
  year   = {2026},
  url    = {https://huggingface.co/rico03/Qwen3.6-27B-Claude-Opus-Reasoning-Distilled}
}

@misc{qwen3.6-27b,
  title  = {{Qwen3.6-27B}: Flagship-Level Coding in a {27B} Dense Model},
  author = {{Qwen Team}},
  month  = {April},
  year   = {2026},
  url    = {https://qwen.ai/blog?id=qwen3.6-27b}
}

🙏 Acknowledgements

• —Jackrong — fine-tuning guide and pipeline this work is based on
• —Unsloth — 2x faster fine-tuning with 70% less VRAM
• —Qwen Team — for releasing Qwen3.6-27B under Apache 2.0
• —All dataset contributors

Released for research and personal use. Not intended for production deployment without additional safety evaluation.