modrill/qwen3-4b-think-baseline-full-sft

Qwen3-4B Code SFT - Think Baseline (Full SFT)

Full-parameter supervised fine-tuning (not LoRA) of Qwen/Qwen3-4B-Base on the think_all dataset with thinking mode enabled.

This repo contains native full fine-tuned weights (single model.safetensors, ~7.5 GB). For LoRA adapters merged into base weights, see modrill/qwen3-4b-think-baseline-lora-sft.

Model Details

• —Base model: Qwen/Qwen3-4B-Base
• —Fine-tuning: Full SFT (DeepSpeed ZeRO-3), finetuning_type: full
• —Dataset: think_all
• —Mode: Think (enable_thinking=true)
• —Training cutoff length: 24576 tokens
• —Epochs: 2
• —Learning rate: 2e-5
• —Train loss: ~0.67
• —Finished: 2026-06-08

Usage

HuggingFace Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "modrill/qwen3-4b-think-baseline-full-sft"
tok = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id, trust_remote_code=True, torch_dtype="auto", device_map="auto"
)

vLLM

python -m vllm.entrypoints.openai.api_server \
  --model modrill/qwen3-4b-think-baseline-full-sft \
  --served-model-name think-baseline-full \
  --port 8801

Inference Tips

• —Set enable_thinking=true in the chat template
• —Recommended max_tokens: 24576

License

Apache 2.0, consistent with the Qwen3 base model license.