NiuNiu0110/RST-SFT-Qwen3.5-27B
RST SFT trajectories for Qwen3.5-27B Multi-turn terminal-agent conversations distilled from Zhongzhi1228/Recursive-Task-Synthesis-Trajectories, ready for supervised fine-tuning of Qwen/Qwen3.5-27B. Pipeline, launchers, and the full plan: https://github.com/k1ssloo/RST-Train cap10 reproduces the paper's SFT example count exactly The source release has 327,189 trajectories. cap10 ends at 10,778 examples — the count arXiv:2608.05466v3 states it trained on. That was… See the full description on the dataset page: https://huggingface.co/datasets/NiuNiu0110/RST-SFT-Qwen3.5-27B.
RST SFT trajectories for Qwen3.5-27B
Multi-turn terminal-agent conversations distilled from Zhongzhi1228/Recursive-Task-Synthesis-Trajectories, ready for supervised fine-tuning of Qwen/Qwen3.5-27B.
Pipeline, launchers, and the full plan: https://github.com/k1ssloo/RST-Train
cap10 reproduces the paper's SFT example count exactly
The source release has 327,189 trajectories. cap10 ends at 10,778 examples — the count arXiv:2608.05466v3 states it trained on. That was not tuned toward; it fell out of the filter chain below. Read it as strong corroboration that the paper used a per-group cap of ~10 with essentially this filtering, not as proof.
327,189 trajectories
├─ gate: status=completed ∧ has_trajectory ∧ ¬has_exception
│ ∧ reward=1.0 ∧ task_present_in_task_dataset
│ → 60,932 clean successes over 1,338 task groups
├─ per-group cap (round-robin across the 4 generating models) → 11,582
├─ reconstruct ATIF-v1.7 → messages, normalize assistant JSON → 11,090
├─ dedup (exact + per-group command signature) → 11,010
└─ drop > 32,768 tokens → 10,778Group-capping is the point: successes per group are median 28, max 284, so uncapped training would be dominated by a handful of lineages.
Schema
messages[0] role=user full Terminus-2 harness prompt + task + initial screen
messages[1] role=assistant canonical JSON {analysis, plan, commands[, task_complete]}
messages[2] role=user terminal observation
...messages[0] is `user`, not `system`, because that is how Terminus-2 delivers the harness prompt (steps[0].source == "user" upstream). Keeping it as user makes training and serving identical — changing it introduces a train/serve skew.
Two processing details that matter
1. Assistant JSON was renormalized. 62.6 % of upstream assistant turns are wrapped in ``` `json `` fences or carry extra prose; 0.1 % are unparseable (dropped). Turns are re-serialized to canonical indent=2` JSON preserving key order.
2. The warning preamble was repaired. When an upstream turn was fenced, the following observation begins Previous response had warnings: - Extra text detected before JSON object. Normalizing the assistant turn without stripping that preamble trains the model to accept "you had warnings" feedback for clean output. 50,169 observations needed this repair in cap10.
cap10_pretokenized: the same data with the mask already applied
Same 10,778 examples, 99,939,485 tokens, 32,402,050 trained tokens (32.42 %). It is smaller than the messages version (75 MB vs 87 MB) because token ids compress better than JSON text.
To get labels, set labels[i] = input_ids[i] where loss_mask[i] == 1 else -100. Do not shift — HuggingFace models shift internally.
Why you may want this instead of messages
Building the mask yourself is the easiest place in this pipeline to be silently wrong: a bad mask still trains, the loss still falls, and the model just comes out worse. Two concrete traps:
1. Do not let a trainer re-tokenize turn-by-turn. verl's MultiTurnSFTDataset templates each message separately and concatenates. Measured on 200 rows of this dataset, 200/200 disagree with the whole-conversation render, because the Qwen3.5 template injects an empty `<think>
</think>
before the **last** assistant turn — so turn-by-turn building makes every turn "last" and a 21-turn conversation ends up with 21 think blocks instead of 1. verl's ignoreinputidsmismatch: True` silences the assertion, not the bug. Using `cap10pretokenized` avoids this entirely.
2. Budget for the logits, not the model. This tokenizer's vocab is 248,320, and the loss upcasts logits to fp32. Measured on one H100-80GB with Qwen3.5-0.8B (0.75 B params!), a real forward/backward over these rows:
At 32,329 tokens the unfused cross-entropy asks for a single 29.85 GiB tensor (seq × 248,320 × 4 bytes) and dies. That term is independent of model size, so a fused/chunked cross-entropy is effectively required at long sequence length regardless of which model you train.
Loss masking
Built for slime's --loss-mask-type qwen3_5, which trains only assistant content. Verified before release: 0 chat-template contract failures, 0 user-turn leakage, 32.6 % of tokens trained. The default --loss-mask-type qwen mis-segments this template and would train on terminal output — do not use it.
Each message may also carry step_loss_mask: 0 to exclude a single assistant turn from the loss while keeping it as context. Unused here; available as a lever.
Limitations
- Reward-verified, not exact-environment-replay-verified. In a 500-sample check only 46 instructions mapped exactly to a public task. These are verifier-passing trajectories; they are not a claim that each was replayed in a byte-identical environment.
- Source mix is skewed toward one iterated model (
qwen35-27b-iter0000161-hf, ~63 %). Rebalance via--modelsin the builder if that matters to you. - 232 trajectories were dropped for exceeding 32,768 tokens, which biases mildly against the longest-horizon episodes.
- Success-only. The 166,660 clean failures are not here; 1,279 groups have both successes and failures and are a ready-made offline preference set.
Attribution
Derived from Zhongzhi1228/Recursive-Task-Synthesis-Trajectories (CC-BY-4.0) by Zhongzhi1228 et al., Recursive Synthesis for Long-Horizon Terminal Tasks (arXiv:2608.05466). Released under the same license. Trajectories were filtered, reconstructed, normalized, deduplicated, and re-serialized; no new rollouts were generated.