Datasetpublic

CoopReason/TESSY-SuperGPQA-3K

TESSY-SuperGPQA-3K 📄 Paper Link    |    🔗 GitHub Repository 📣 Paper 🎉 Accepted at ICML 2026! How to Fine-Tune a Reasoning Model? A Teacher–Student Cooperation Framework to Synthesize Student-Consistent SFT Data 🚀 Overview We construct a programming contest training dataset for Qwen3-8B by leveraging GPT-OSS-120B as the teacher model. The synthesized data preserves the strong reasoning capabilities of… See the full description on the dataset page: https://huggingface.co/datasets/CoopReason/TESSY-SuperGPQA-3K.

sourceHugging Faceapache-2.0updated 5mo agoView on Hugging Face

6likes157downloads

Dataset Card

TESSY-SuperGPQA-3K

<p align="center"> 📄 <a href="https://arxiv.org/pdf/2604.14164">Paper Link</a>    |    🔗 <a href="https://github.com/CoopReason/TESSY/tree/main">GitHub Repository</a> </p>

📣 Paper

🎉 Accepted at ICML 2026!

How to Fine-Tune a Reasoning Model? A Teacher–Student Cooperation Framework to Synthesize Student-Consistent SFT Data

🚀 Overview

We construct a programming contest training dataset for Qwen3-8B by leveraging GPT-OSS-120B as the teacher model. The synthesized data preserves the strong reasoning capabilities of GPT-OSS-120B, while being aligned with the data distribution of Qwen3-8B. This enables more effective on-policy SFT, ensuring that the student model learns from samples that are both high-quality and distribution-consistent with its own generation behavior.

Note: This dataset is specifically tailored and optimized for Qwen/Qwen3-8B. We use GPT-OSS-120B as teacher and Qwen3-8B as student to synthesize this dataset. The question is sampled from m-a-p/SuperGPQA.

💡 Motivation

Training reasoning models (e.g., Qwen3) is highly sensitive to the data distribution. We observe that:

❗ Using off-policy data (e.g., directly from a strong teacher model) for SFT can lead to severe catastrophic forgetting, especially for complex reasoning tasks.

🔦 Key Idea

To address this critical issue, we propose TESSY, a novel Teacher–Student Cooperative Data Synthesis framework designed to generate on-policy training data. Instead of relying on a teacher model to fully generate training samples, TESSY decouples the generation process into two distinct parts:

—🧠 Teacher model → specializes in generating capability tokens.
—✍️ Student model → focuses on generating style tokens (e.g., Hmm, Wait...).

This cooperative approach ensures:

—Alignment with student distribution (on-policy): The synthesized data is tailored to the student model's own generation patterns.
—Preservation of teacher reasoning quality: The teacher's advanced reasoning capabilities are effectively leveraged and maintained.

🧩 Method

TESSY performs iterative cooperative generation through the following steps:

1.Predict Reasoning Boundaries: The process begins by identifying the boundaries between reasoning steps and non-reasoning content within a given problem.
2.Alternate Generation: The teacher and student models then alternate in generating parts of the solution.
3.Construct Full Trajectories: By combining these collaboratively generated segments, TESSY constructs complete, high-quality reasoning trajectories that are aligned with the student model's distribution.

⚙️ Download

Install package datasets:

bash

pip install datasets

Load this dataset:

python

from datasets import load_dataset

dataset = load_dataset("CoopReason/TESSY-SuperGPQA-3K", split="train")

print(dataset[0]["dialogs"][0]["content"])

# Output: question

print(dataset[0]["dialogs"][1]["content"])

# Output: Synthetic reasoning trajectory and final answer

📌 Citation

If this work is useful to you, please cite:

bibtex

@article{TESSY,
  title={How to Fine-Tune a Reasoning Model? A Teacher-Student Cooperation Framework to Synthesize Student-Consistent SFT Data},
  author={Huang, Zixian and Yang, Kaichen and Huang, Xu and Hao, Feiyang and Ge, Qiming and Li, Bowen and Du, He and Chen, Kai and Guo, Qipeng},
  journal={arXiv preprint arXiv:2604.14164},
  year={2026}
}