Modelpublic

lllqaq/Qwen2.5-Coder-7B-SWEsmith-400

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face

0likes19downloads

Model Card

Qwen2.5-Coder-7B-SWEsmith-400

A full fine-tune of Qwen/Qwen2.5-Coder-7B-Instruct on 400 SWE-agent trajectories from the SWE-smith project, intended for use as a software-engineering agent (SWE-agent / SWE-bench style tasks).

⚠️ Experimental / small-scale. This was trained on only 400 trajectories as a workflow validation run, not a production model. Treat it as a reproducibility / sanity baseline.

Training data

—Source: `SWE-bench/SWE-smith-trajectories`, xml split (the format SWE-agent-LM was trained on).
—Subset: first 400 of the 26,076 trajectories (multi-turn agent rollouts, avg ~67 messages/traj).
—Format: OpenAI-style messages, supervised on assistant turns only (train_on_input=False).

Method

Full-parameter SFT with torchtune (full_finetune_distributed), using SWE-smith's default 7B config.

Hyperparameter	Value
Base model	Qwen2.5-Coder-7B-Instruct
Optimizer	AdamW (fused), wd=0.01
LR / schedule	1e-4, cosine, 5 warmup steps
Epochs	3
Max seq len	32768
Precision	bf16
Per-device batch	1, grad-accum 4
Hardware	7× A100 80GB (FSDP) → effective batch 28
Total optimizer steps	42
Activation checkpointing	on

Training loss: 0.41 → 0.19 (ep0) → 0.11 (ep1) → 0.085 (ep2). This repo is the epoch 2 checkpoint.

Intended use & limitations

Designed to be served (vLLM / SGLang) behind SWE-agent for automated bug-fixing on repository-level tasks. Given the tiny training set, expect limited generalization; it has not been evaluated on SWE-bench. Inherits the base model's capabilities and biases.

Citation

Built with SWE-smith (Yang et al., NeurIPS 2025). Base model license: Apache-2.0.