keypa/Qwen3.5-9B-Claude-Opus-4.7-GGUF
Qwen3.5-9B-Claude-Opus-4.7-GGUF
Description
This repo contains GGUF weights for the fine-tuned Qwen3.5-9B-Claude-Opus-4.7. This is a 9-billion parameter model distilled from advanced reasoning chains generated by Claude Opus 4.7, optimized for high-level logical deduction and step-by-step problem solving.
As a Vision-Language-Model (VLM), it retains the base model's ability to analyze images while applying the enhanced reasoning logic learned during the fine-tuning process.
Available Files
For the best balance of performance and intelligence, we recommend the Q4_K_M or Q5_K_M versions.
- Qwen3.5-9B.Q4_K_M.gguf: Recommended for most users (4-bit quantization).
- Qwen3.5-9B.Q5_K_M.gguf: High fidelity (5-bit quantization).
- Qwen3.5-9B.Q8_0.gguf: Near-perfect precision (8-bit quantization).
- Qwen3.5-9B.F16.gguf: Full weights (16-bit).
- Qwen3.5-9B.BF16-mmproj.gguf: Required for Vision functionality. Use this alongside any of the GGUF files above to enable image analysis.
Usage Instructions
1. Multimodal (Vision + Text)
To use the model with images, you must provide the multimodal projector (mmproj) file:
llama-mtmd-cli -hf keypa/Qwen3.5-9B-Claude-Opus-4.7-GGUF --mmproj Qwen3.5-9B.BF16-mmproj.gguf --jinja2. Text-Only Reasoning
For standard logic and chat tasks:
llama-cli -hf keypa/Qwen3.5-9B-Claude-Opus-4.7-GGUF --jinja3. Prompting for Reasoning
This model was trained to think before it speaks. For best results, use the following template structure:
<|im_start|>system
You are a helpful assistant with advanced reasoning capabilities.<|im_end|>
<|im_start|>user
[Your Question or Image Here]<|im_end|>
<|im_start|>assistant
<|im_thought|>Technical Details
- Finetuned from: Qwen/Qwen3.5-9B
- Training Method: QLoRA via Unsloth
- Dataset:lordx64/reasoning-distill-claude-opus-4-7-max (Distilled from Claude Opus 4.7)
- Conversion: Merged to 16-bit and quantized via
llama.cpp
This model was trained 2x faster with Unsloth.