CoolFace
Datasetpublic

JackHsieh/luna-reason-only.stride-train8-test32.k-8.statml-arxiv.qwen3-ids.tags

luna-reason-only.stride-train8-test32.k-8.statml-arxiv.qwen3-ids.tags A pre-tokenized, tag-wrapped variant of JackHsieh/luna-reason-only.stride-train8-test32.k-8.statml-arxiv.qwen3-ids. Each thought_text is wrapped as <|note|>{thought_text}<|/note|> and stored both as text (thought_text) and as Qwen/Qwen3-4B-Base token ids (input_ids). Tag ids and the spliced key are inserted as ids, never re-tokenized. Delimiter ids: <|note|> = 151669, <|/note|> = 151670. These are the first… See the full description on the dataset page: https://huggingface.co/datasets/JackHsieh/luna-reason-only.stride-train8-test32.k-8.statml-arxiv.qwen3-ids.tags.

sourceHugging Facecc-by-4.0updated 28d agoView on Hugging Face
0likes46downloads
Dataset Card

luna-reason-only.stride-train8-test32.k-8.statml-arxiv.qwen3-ids.tags

A pre-tokenized, tag-wrapped variant of `JackHsieh/luna-reason-only.stride-train8-test32.k-8.statml-arxiv.qwen3-ids`. Each thought_text is wrapped as

<|note|>{thought_text}<|/note|>

and stored both as text (thought_text) and as Qwen/Qwen3-4B-Base token ids (input_ids). Tag ids and the spliced key are inserted as ids, never re-tokenized.

Delimiter ids: <|note|> = 151669, <|/note|> = 151670. These are the first two special token ids under the Qwen3 tokenizer; a training run must declare them under trainee.special_tokens in this order.

Thought lengths in Qwen3 tokens (input_ids length, tags included)

**split****thoughts****mean ± std****[min, max]**
train612_864310.2 ± 56.5[74, 771]
test72_960311.8 ± 56.2[105, 542]
combined685_824310.4 ± 56.5[74, 771]