Modelpublic

davidanugraha/Qwen3.5-9B-SWE-Smith-LoRA-Adapters

sourceHugging Faceapache-2.0updated 16d agoView on Hugging Face

0likes

Model Card

Qwen3.5-9B SWE-Smith LoRA adapters

This repository contains adapter-only checkpoints and reproducibility records for the verl-qwen35-9b-65k-mini-swe-rloo-20260830-r15 → verl-qwen35-9b-65k-mini-swe-rloo-20260831-r16 continuation lineage. It does not contain the Qwen3.5-9B base-model weights. Use Qwen/Qwen3.5-9B at revision c202236235762e1c871ad0ccb60c8ee5ba337b9a.

Adapter checkpoints

—checkpoints/step-10
—checkpoints/step-20
—checkpoints/step-30
—checkpoints/step-40
—checkpoints/step-50
—checkpoints/step-60
—checkpoints/step-70
—checkpoints/step-80
—checkpoints/step-90
—checkpoints/step-100
—checkpoints/step-110
—checkpoints/step-120

The training plan targeted 143 optimizer steps, but the latest committed checkpoint is step 120; no steps 130, 140, or 143 exist in the durable store. Step 10 belongs to r15, and steps 20–120 belong to the resumed r16 run.

Each adapter directory contains adapter_config.json, adapter_model.safetensors, and export_provenance.json.

Resuming training

continuation/global_step_120/ is the complete committed veRL/FSDP checkpoint, not merely a serving adapter. It contains all eight model shards, all eight optimizer shards, all eight extra/RNG-state shards, data.pt, FSDP/LoRA metadata, tokenizer metadata, and the original checkpoint manifest. The validated continuation payload is 1258355426 bytes across 32 manifest-listed files.

The raw step-120 model shards contain the trained LoRA state needed by veRL; they are not Qwen base-model weights. Restore against the exact base revision, veRL commit, training image, and run identity recorded under training/ and provenance/.

Training contract

—Algorithm: RLOO
—LoRA: rank 32, alpha 64
—Batch: 32 prompts × 8 rollouts
—Context: 65,536 tokens; maximum generated response 57,344 tokens
—Sampling: temperature 1.0, top-p 1.0, top-k -1
—Agent: Mini-SWE-Agent
—Sequence parallelism: 8
—veRL commit and immutable training-image digest are recorded in the run identities

SWE-Bench Verified records

These are the completed evaluation records found for this lineage. Protocols differ, so rows with different sampling or retry policies should not be compared as a clean checkpoint sweep.

Arm	Protocol	Resolved	Score	Agent timeouts
`base-temp0.6`	temperature 0.6; retry-aware	235/500	47.0%	37
`base-temp1.0`	temperature 1.0; retry-aware	260/500	52.0%	225
`step20-temp0.6`	temperature 0.6; retry-aware	237/500	47.4%	49
`step20-temp1.0`	training sampler; temperature 1.0; retry-aware	254/500	50.8%	191
`step40-temp1.0`	training sampler; temperature 1.0; retry-aware	249/500	49.8%	193
`step60-temp1.0-no-agent-timeout-retry`	training sampler; temperature 1.0; no agent-timeout retry	199/500	39.8%	160

The full checksummed evaluation artifacts are under evaluation/artifacts/.

Provenance

—training/runs/r15/ and training/runs/r16/ contain the original run identities, Hydra configuration, preflight evidence, and checkpoint manifests.
—training/wandb/ is the r16 W&B export; training/resume-run/wandb/ is r15.
—provenance/training-image/ is extracted from the immutable digest-pinned r16 training image and is the authoritative historical runtime source.
—provenance/repository-at-export/ is a convenience snapshot of the current export, launch, and evaluation tooling and is not asserted to match the historical image.

Loading an adapter

For example, with a LoRA-enabled vLLM server:

bash

vllm serve Qwen/Qwen3.5-9B \
  --revision c202236235762e1c871ad0ccb60c8ee5ba337b9a \
  --served-model-name Qwen3.5-9B \
  --max-model-len 65536 \
  --enable-prefix-caching \
  --enable-lora --max-lora-rank 64 \
  --lora-modules r16-step-120=/path/to/repo/checkpoints/step-120