jamesdborin/SPEED-Bench-Qualitative-Qwen3.6-35B-A3B-FP8-torchspec

SPEED-Bench Qualitative Qwen3.6 TorchSpec

TorchSpec-compatible chat dataset generated from the 880 fully materialized SPEED-Bench qualitative prompts.

Responses were generated on Doubleword with Qwen/Qwen3.6-35B-A3B-FP8 using /v1/chat/completions and max_tokens=4096.

Files

• —data/train.jsonl: 880 rows in TorchSpec chat format.

Schema

Each row contains:

{
  "id": "<speedbench_question_id>",
  "conversations": [
    {"role": "user", "content": "<prompt>"},
    {
      "role": "assistant",
      "reasoning_content": "<model reasoning, when returned>",
      "content": "<final answer, when returned>"
    }
  ],
  "metadata": {
    "source": "nvidia/SPEED-Bench qualitative",
    "question_id": "<speedbench_question_id>",
    "model": "Qwen/Qwen3.6-35B-A3B-FP8",
    "batch_id": "b76696bf-2814-4834-b5fe-e4b12efe8c72",
    "finish_reason": "stop|length",
    "has_reasoning_content": true,
    "has_final_content": true,
    "prompt_tokens": 0,
    "completion_tokens": 0,
    "total_tokens": 0
  }
}

Qwen Reasoning Formatting

Qwen/Qwen3.6-35B-A3B-FP8 uses Qwen chat tokens plus explicit thinking tags. Its tokenizer renders assistant reasoning as:

<|im_start|>assistant
<think>
...reasoning_content...
</think>

...content...<|im_end|>

This dataset therefore stores reasoning_content as a separate assistant-message field instead of manually concatenating it into content. TorchSpec preserves this field when loading conversations, and the Qwen tokenizer chat template inserts the correct <think> / </think> tags during formatting.

Recommended TorchSpec overrides:

dataset.train_data_path=jamesdborin/SPEED-Bench-Qualitative-Qwen3.6-35B-A3B-FP8-torchspec \
dataset.prompt_key=conversations \
dataset.chat_template=qwen

Use dataset.chat_template=qwen with the Qwen3.6 tokenizer for this dataset. The model tokenizer handles reasoning_content natively. Avoid flattening reasoning into plain answer text.

Generation Notes

• —Rows: 880
• —Finish reasons: length=290, stop=590
• —Assistant fields: contentandreasoning=666, reasoning_only=214
• —Completions at the 4096 token cap: 290

For 214 rows, generation reached the token cap while still in reasoning, so content is empty and reasoning_content contains the generated text. These rows are retained because the tokenizer still renders them as supervised assistant reasoning under <think>...</think>.