CoolFace
Datasetpublic

kmoss/qwen3-1.7b-traces

Qwen3-1.7B reasoning traces On-policy chain-of-thought rollouts from Qwen/Qwen3-1.7B in thinking mode, collected to study long-context KV-cache residency constraints (trainable sparse attention, in the InfLLM-V2 / NOSA line). Used as adaptation data: training a sparse+local attention variant on the model's own output distribution avoids the alignment tax that continued pretraining on external corpora imposes on a post-trained model. Caveats Not filtered for… See the full description on the dataset page: https://huggingface.co/datasets/kmoss/qwen3-1.7b-traces.

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face
0likes35downloads
Dataset Card

Qwen3-1.7B reasoning traces

On-policy chain-of-thought rollouts from `Qwen/Qwen3-1.7B` in thinking mode, collected to study long-context KV-cache residency constraints (trainable sparse attention, in the InfLLM-V2 / NOSA line).

Used as adaptation data: training a sparse+local attention variant on the model's own output distribution avoids the alignment tax that continued pretraining on external corpora imposes on a post-trained model.

Caveats

  • —Not filtered for correctness. Answers are unverified and many are wrong. The purpose is on-distribution rollout generation, not solution quality. Do not treat these as ground truth.
  • —Model-generated text: expect reasoning errors, dead ends, and self-contradiction. That is part of the distribution being modelled.
  • —Sampling: temperature=0.6, top_p=0.95, top_k=20 (Qwen3's official thinking-mode settings), max_model_len=32768, max_tokens set per prompt to 32768 - prompt_tokens - 64 so traces run to natural length rather than a fixed cap.

Source benchmarks

Only the problem statements are used.

`bench`sourceitems x samples
aime24Maxwell-Jia/AIME_202430 x 8
aime25yentinglin/aime_202530 x 8
math500HuggingFaceH4/MATH-500500 x 1
zebralogicallenai/ZebraLogicBench (grid_mode)200 x 1
autologiqzhu/AutoLogi (English subset)200 x 1

Schema

bench, idx, sample, text (full completion incl. <think>...</think>), token_ids (Qwen3 tokenizer), prompt_tokens, completion_tokens, total_tokens, think_tokens, answer_tokens, finish_reason, truncated, meta (JSON string of benchmark-specific fields).

python
from datasets import load_dataset
ds = load_dataset("kmoss/qwen3-1.7b-traces", split="train")