Datasetpublic

oldpilluwu/openthoughts3-qwen3-8b-300k

OpenThoughts3 300K — Qwen3-8B SFT Trajectories This dataset contains 300,000 synthetic reasoning trajectories generated by Qwen/Qwen3-8B from prompts sampled from open-thoughts/OpenThoughts3-1.2M. It was prepared for the supervised fine-tuning stage of the Lightning OPD Qwen3-4B experiment, where Qwen/Qwen3-4B-Base is the student and Qwen3-8B is the teacher. Dataset construction Setting Value Prompt source open-thoughts/OpenThoughts3-1.2M, train split… See the full description on the dataset page: https://huggingface.co/datasets/oldpilluwu/openthoughts3-qwen3-8b-300k.

sourceHugging Facemitupdated 2mo agoView on Hugging Face

0likes32downloads

Dataset Card

OpenThoughts3 300K — Qwen3-8B SFT Trajectories

This dataset contains 300,000 synthetic reasoning trajectories generated by `Qwen/Qwen3-8B` from prompts sampled from `open-thoughts/OpenThoughts3-1.2M`. It was prepared for the supervised fine-tuning stage of the Lightning OPD Qwen3-4B experiment, where Qwen/Qwen3-4B-Base is the student and Qwen3-8B is the teacher.

Dataset construction

Setting	Value
Prompt source	`open-thoughts/OpenThoughts3-1.2M`, train split
Prompt selection	300,000 source rows sampled with Python seed 42
Teacher	`Qwen/Qwen3-8B`
Responses per sampled row	1
Temperature	0.7
Top-p	0.9
Maximum new tokens	16,384
Thinking mode	Enabled
Intended student	`Qwen/Qwen3-4B-Base`

The prompts were sampled before removing the original assistant responses. OpenThoughts3 contains multiple source trajectories for some questions, so the resulting dataset intentionally includes repeated prompt text.

Generation is stochastic. The prompt subset and ordering follow the seed-42 Lightning-OPD preparation recipe, but the generated responses are not expected to be byte-identical to another run of the same pipeline. The Parquet file does not record the exact teacher revision, inference-engine version, or generation RNG state.

Schema

The dataset contains a single train split with two columns:

Column	Type	Description
`messages`	list of `{role, content}`	One user message followed by one Qwen3-8B assistant response
`tokens`	int64	Number of generated assistant tokens reported by the inference engine

Example structure:

python

{
    "messages": [
        {"role": "user", "content": "..."},
        {"role": "assistant", "content": "<think>\n...\n</think>\n\n..."},
    ],
    "tokens": 11782,
}

Audited statistics

Metric	Value
Rows	300,000
Parquet size	4,713,439,990 bytes (4.39 GiB)
Total assistant tokens	4,304,723,562
Mean assistant tokens	14,349.1
Median assistant tokens	16,384
Unique prompt texts	64,454
Exact duplicate conversations	31
Responses at the 16,384-token generation ceiling	182,000 (60.7%)
Responses without a closing `</think>` tag	167,014 (55.7%)

All 300,000 rows have the expected user → assistant role structure. No null or empty message fields were found during the audit.

Usage

python

from datasets import load_dataset

dataset = load_dataset(
    "oldpilluwu/openthoughts3-qwen3-8b-300k",
    split="train",
)

print(dataset[0]["messages"])
print(dataset[0]["tokens"])

For the paper-style SFT stage, apply the Qwen3 chat template, train with assistant-token loss, enable sequence packing, and use a 16,384-token cutoff. The released Lightning-OPD recipe uses full-parameter SFT for 3,000 steps with a global batch size of 256, learning rate 8e-5, cosine decay, and a 0.1 warmup ratio.

Lightning-OPD compatibility

Lightning OPD requires teacher consistency: the teacher used to produce the SFT trajectories must also be used to compute teacher log-probabilities during the later OPD stage. To follow the 4B experiment, use the same Qwen3-8B checkpoint and revision in both stages.

References

bibtex

@article{wu2026lightning,
  title={Lightning OPD: Efficient On-Policy Distillation for Large Reasoning Models without Live Teacher Serving},
  author={Wu, Yecheng and Han, Song and Cai, Han},
  year={2026}
}

License

This repository is published under the MIT license metadata shown above. Users are also responsible for complying with the licenses and terms of the source prompt dataset and teacher model.