JackHsieh/4B-Instruct-RL-step2176-reason-only.stride-1.k-8.statml-arxiv.qwen3-ids.kv-tags-explained

4B-Instruct-RL-step2176-reason-only.stride-1.k-8.statml-arxiv.qwen3-ids.kv-tags-explained

Tokenized, tag-wrapped form of `JackHsieh/4B-Instruct-RL-step2176-reason-only.stride-1.k-8.statml-arxiv.qwen3-ids`, the thoughts written by the prestar-RL policy `Qwen3-4B-Instruct-2507.prestar-RL.reason-only.lr7e-7-kl0.step-2176`. Each thought is wrapped as

<|note|>
This is a hint about a span that appears later in this document. KEY is the text immediately before that span; VALUE is a note about what might come next.
KEY: <last 8 prefix tokens>
VALUE: <thought>
<|/note|>

and stored both as text (thought_text) and as Qwen/Qwen3-4B-Base token ids (input_ids).

Longest thought: 1070 tokens — a training run's max_thought_length must be at least this.

Delimiter ids: <|note|> = 151669, <|/note|> = 151670. A run must declare these under trainee.special_tokens IN THIS ORDER, or the ids will not match.

Intended to be PREPENDED to the document rather than interjected between prefix and suffix: the key names the span the value is about. The key is the 8 document tokens ending at the cut (input_ids[chunk_start_index - 8:chunk_start_index]), spliced as ids rather than re-tokenized.

Provenance

Source thoughts: JackHsieh/4B-Instruct-RL-step2176-reason-only.stride-1.k-8.statml-arxiv.qwen3-ids. Documents: JackHsieh/statML-arxiv-40M-20M.

Thoughts the generator cut off at its 1,024-token cap (380 in train, 18 in test) carry the placeholder VALUE No hint was produced for this key. instead of the partial text, mirroring how the RL reward treats a thought that never closes. Tokenizer: Qwen/Qwen3-4B-Base, add_special_tokens=False; tag ids and any document ids are inserted explicitly, so no BOS is introduced and no token is round-tripped through text.

Schema

All source columns, plus/with:

Token counts (input_ids length, tags included)

Nothing is trimmed: 12,339,112 thoughts exceed 384 tokens, so a run with a shorter max_thought_length must trim them (at a sentence boundary) or drop those chunks.