ubicloud/filtered-SWE-Openhands-Devstral-32k-20K
Dataset Summary This dataset contains 20,000 high-quality SFT trajectories filtered and selected from SWE-Star, a 250K+ sample dataset collected under a modified OpenHands scaffold. The data is used to train SWE agent models under limited resources with a 32K context window. A single-source dataset is intentionally selected so that all training trajectories share a uniform scaffold, isolating the scaffold inconsistency effect: any performance variation across evaluation… See the full description on the dataset page: https://huggingface.co/datasets/ubicloud/filtered-SWE-Openhands-Devstral-32k-20K.
Dataset Summary
This dataset contains 20,000 high-quality SFT trajectories filtered and selected from SWE-Star, a 250K+ sample dataset collected under a modified OpenHands scaffold. The data is used to train SWE agent models under limited resources with a 32K context window.
A single-source dataset is intentionally selected so that all training trajectories share a uniform scaffold, isolating the scaffold inconsistency effect: any performance variation across evaluation scaffolds can be attributed to the training-inference scaffold mismatch rather than mixed training signals.
Statistics
Data Columns
Data Filtering
10 hard filtering rules organized by behavioral issues:
Structural Validity
Reject trajectories with zero edits, fewer than 3 assistant turns, missing or excessively short system prompts (<100 chars), or empty assistant messages. These are basic data integrity checks: broken records introduce noise rather than signal during SFT.
Exploration Before Action
Require at least one view or search operation before the first edit, and cap reproduction-script edits at 50% of all edits. Small models have limited step budgets and must learn to understand before modifying; trajectories that edit blindly or spend most turns on diagnostic scripts teach poor resource allocation.
Edit Grounding
Compute a composite edit integrity score (0-1) penalizing:
- Blind edits (editing unread files)
- Weak anchors (
old_strtoo short relative tonew_str) - Undefined references (using names never seen in context)
Trajectories scoring below 0.3 are rejected, as are those with ≥3 consecutive edit-then-error cycles. Small models are prone to imitating surface patterns; these rules ensure they only see edits anchored in observed repository state.
Verification and Efficiency
Require at least one test or verification command after the final edit, and reject trajectories with ≥3 consecutive identical command repetitions. In a limited-step setting, every wasted turn reduces success probability; these rules ensure the model learns to verify its work and avoid repetitive loops.
Data Selection
After filtering, trajectories are scored and ranked to select 20K samples for SFT using a quality-first with diversity as a soft constraint strategy. Trajectories are ranked by a weighted composite score, then greedily selected while penalizing over-representation along three axes (repository, tool pattern, step-count bucket).
Composite Score Dimensions
Selection proceeds in two passes: a feature-extraction pass that scores all filtered records, followed by a greedy pass that selects 20K from a 3× buffer (60K candidates) while applying diversity penalties.
Use Cases
- SFT fine-tuning: Train SWE agent models on uniform scaffold data under limited resources
- Agent research: Study scaffold mismatch effects and data quality impact on agent performance
- Data analysis: Analyze efficient debugging and code repair strategies at scale
Source Data
Filtered and selected from SWE-Star.