CoolFace
Modelpublic

AlienKevin/SWE-ZERO-100K-Qwen3-1.7B-Base

sourceHugging Faceapache-2.0updated 5mo agoView on Hugging Face
0likes13downloads
Model Card

SWE-ZERO-100K-Qwen3-1.7B-Base

Qwen3-1.7B-Base SFT on a 100K random sample of SWE-ZERO trajectories (superset of the 10K, right-truncated to 8K tokens), evaluated on SWE-bench Verified.

Eval result

pass@1 = 9/100 = 9% on the 100-task SWE-bench Verified slice (latest trial per task) — beats 10K @ 7%.

For full eval details + per-task trajectories, see the eval dataset: AlienKevin/SWE-ZERO-100K-Qwen3-1.7B-Base-eval.

Training

  • —Base: Qwen/Qwen3-1.7B-Base
  • —SFT data: 100K random sample from AlienKevin/SWE-ZERO-12M-trajectories @ 2f328e1d (superset of the 10K), right-truncated to 8K tokens
  • —Model arch: max_seq_len=32768 (Llama 3 RoPE scaling from 8192)
  • —TPU: v5p-16 (~6.5h training)
  • —Optimizer: AdamW (β1=0.9, β2=0.95, ε=1e-8), lr=2e-5, weightdecay=0.1, maxgradnorm=30, cosine schedule, warmup=0.03, minlr_ratio=0.1
  • —Batch: 16 global, 6249 steps
  • —Tracking: marin#5611

Critical detail: eos_token_id

The HF config has eos_token_id: [151643, 151645] so vLLM stops at both <|endoftext|> AND <|im_end|> (Qwen3 chat-template turn boundary).

Inference

python
from vllm import LLM, SamplingParams

llm = LLM(model="AlienKevin/SWE-ZERO-100K-Qwen3-1.7B-Base", max_model_len=32768)
params = SamplingParams(temperature=1.0, max_tokens=4096)

See the eval dataset for the harbor + mini-swe-agent v1 config used in our results.