Modelpublic

OpenHands/CodeScout-14B

sourceHugging Faceapache-2.0updated 6mo agoView on Hugging Face

4likes122downloads

Model Card

CodeScout-14B

Strongest CodeScout model — open-source SOTA on SWE-Bench code localization.

CodeScout-14B is part of the CodeScout family of open-source RL-trained code search agents. CodeScout models achieve state-of-the-art repository-level code localization using nothing more than a standard Unix terminal — no static analysis, no repository graphs, no language-specific tooling.

Key Highlights

—Open-source SOTA on SWE-Bench Verified, Pro, and Lite for code localization
—Outperforms 2–18× larger base and post-trained LLMs across all benchmarks
—Surpasses GPT-5 and approaches Claude Sonnet 4.5 using RepoNavigator, despite using only a bash terminal
—Achieves 8–33% higher function-level F1 than Qwen3-32B (Thinking)

Results

Performance on SWE-Bench code localization (instance-averaged F1 scores):

Benchmark	CodeScout-1.7B	CodeScout-4B	CodeScout-14B
SWE-Bench Verified — File F1	55.46	68.52	68.57
SWE-Bench Verified — Func F1	28.22	36.78	40.32
SWE-Bench Pro — File F1	40.96	51.77	53.63
SWE-Bench Pro — Func F1	18.24	29.03	28.74
SWE-Bench Lite — File F1	56.57	67.03	71.84
SWE-Bench Lite — Func F1	27.07	39.87	44.43

Code localization performance on SWE-Bench Verified. CodeScout (⭐) achieves superior or competitive results over larger open-source LLMs and narrows the gap with closed-source frontier models.

Training

CodeScout-14B is trained directly from Qwen3-14B using GSPO (Group Sequence Policy Optimization) reinforcement learning.

—Training data: 9,600 instances from SWE-Smith (39K filtered, 128 repos)
—RL steps: 300
—Batch size: 32, with 4 rollouts per instance
—Max context length: 50K tokens (extended with YaRN)
—Max turns per episode: 4
—Reward: Multi-level F1 (file + module + function) with auxiliary turn-completion reward
—Hardware: 8×H100 GPUs
—Learning rate: 1e-6 (constant)

How It Works

CodeScout uses the OpenHands-Bash scaffold — an agent equipped with only a Terminal tool (supporting standard Unix commands like rg, find, grep, ls) and a LocalizationFinish tool for structured output submission. The agent iteratively navigates the repository to identify relevant files, classes, and functions related to a given issue.

The model is trained with GSPO (Group Sequence Policy Optimization) using multi-level F1 rewards at the file, module, and function level.

Intended Use

CodeScout-14B is designed for repository-level code localization: given a GitHub issue description and a code repository, it identifies the relevant files, classes, and functions that need to be modified. It is intended to be used as a localization subagent within larger coding agent pipelines.

Limitations

—Trained and evaluated exclusively on Python repositories
—Designed for code localization, not code editing or issue resolution
—Performance may vary on repositories significantly different from the training distribution
—Requires the OpenHands-Bash scaffold for optimal performance

Citation

bibtex

@misc{sutawika2026codescouteffectiverecipereinforcement,
      title={CodeScout: An Effective Recipe for Reinforcement Learning of Code Search Agents}, 
      author={Lintang Sutawika and Aditya Bharat Soni and Bharath Sriraam R R and Apurva Gandhi and Taha Yassine and Sanidhya Vijayvargiya and Yuchen Li and Xuhui Zhou and Yilin Zhang and Leander Melroy Maben and Graham Neubig},
      year={2026},
      eprint={2603.17829},
      archivePrefix={arXiv},
      primaryClass={cs.SE},
      url={https://arxiv.org/abs/2603.17829}, 
}