JackHsieh/luna-reason-only.stride-train8-test32.k-8.statml-arxiv.qwen3-ids.kv-tags-explained
luna-reason-only.stride-train8-test32.k-8.statml-arxiv.qwen3-ids.kv-tags-explained A pre-tokenized, tag-wrapped variant of JackHsieh/luna-reason-only.stride-train8-test32.k-8.statml-arxiv.qwen3-ids. Each thought_text is wrapped as <|note|> This is a hint about a span that appears later in this document. KEY is the text immediately before that span; VALUE is a note about what might come next. KEY: {last 8 prefix tokens} VALUE: {thought_text} <|/note|> and stored both as text… See the full description on the dataset page: https://huggingface.co/datasets/JackHsieh/luna-reason-only.stride-train8-test32.k-8.statml-arxiv.qwen3-ids.kv-tags-explained.
luna-reason-only.stride-train8-test32.k-8.statml-arxiv.qwen3-ids.kv-tags-explained
A pre-tokenized, tag-wrapped variant of `JackHsieh/luna-reason-only.stride-train8-test32.k-8.statml-arxiv.qwen3-ids`. Each thought_text is wrapped as
<|note|>
This is a hint about a span that appears later in this document. KEY is the text immediately before that span; VALUE is a note about what might come next.
KEY: {last 8 prefix tokens}
VALUE: {thought_text}
<|/note|>and stored both as text (thought_text) and as Qwen/Qwen3-4B-Base token ids (input_ids). Tag ids and the spliced key are inserted as ids, never re-tokenized.
Delimiter ids: <|note|> = 151669, <|/note|> = 151670. These are the first two special token ids under the Qwen3 tokenizer; a training run must declare them under trainee.special_tokens in this order.
Intended to be prepended to the document rather than interjected between prefix and suffix: the KEY names the span the VALUE is about. The KEY is the 8 document tokens ending at the cut (input_ids[chunk_start_index - 8 : chunk_start_index]).