Datasetpublic

Xiaofeng77/answer-only-gp-l-only-10k

Debunk the Myth of SFT Generalization Dataset This dataset is associated with the paper "Debunk the Myth of SFT Generalization". The paper challenges the prevailing view that supervised fine-tuning (SFT) primarily memorizes training data and fails to generalize, in contrast to reinforcement learning (RL). It demonstrates that SFT can generalize as well as—or better than—RL when trained with appropriate data, achieved through prompt diversity and Chain-of-Thought (CoT)… See the full description on the dataset page: https://huggingface.co/datasets/Xiaofeng77/answer-only-gp-l-only-10k.

sourceHugging Faceupdated 1y agoView on Hugging Face

0likes52downloads

Dataset Card

Debunk the Myth of SFT Generalization Dataset

This dataset is associated with the paper "Debunk the Myth of SFT Generalization". The paper challenges the prevailing view that supervised fine-tuning (SFT) primarily memorizes training data and fails to generalize, in contrast to reinforcement learning (RL). It demonstrates that SFT can generalize as well as—or better than—RL when trained with appropriate data, achieved through prompt diversity and Chain-of-Thought (CoT) supervision on decision-making benchmarks like Sokoban and General Points.

Code reproducing the results in the paper can be found at: https://github.com/XiaofengLin7/debunking-sft-generalization.

Dataset

This dataset is part of a collection used in the paper's experiments, providing various configurations for evaluating SFT and RL models. The table below outlines these specific datasets:

Task	Method	Diversity	Format	Link
Sokoban	RL	non-diverse	—	🤗
Sokoban	RL	diverse	—	🤗
Sokoban	SFT	non-diverse	answer-only	🤗
Sokoban	SFT	diverse	answer-only	🤗
Sokoban	SFT	non-diverse	cot	🤗
Sokoban	SFT	diverse	cot	🤗
General Points	RL	non-diverse	—	🤗
General Points	RL	diverse	—	🤗
General Points	SFT	non-diverse	answer-only	🤗
General Points	SFT	diverse	answer-only	🤗
General Points	SFT	non-diverse	cot	🤗
General Points	SFT	diverse	cot	🤗

The dataset contains question and answer fields, along with extra_info that provides specific details about the task, such as cards, solution, and target for the General Points benchmark.

Sample Usage

To train your model using the SFT or GRPO methods with the associated code and similar data configurations, you can use the following bash commands from the GitHub repository. Ensure your model and data paths are correctly specified beforehand.

Train your model with SFT

For Sokoban:

bash

bash debunk_sft/scripts/sokoban/sokoban_train_and_eval.sh

For General Points:

bash

bash debunk_sft/scripts/gp_l/gp_l_train_and_eval.sh

Train your model with GRPO

For Sokoban:

bash

bash debunk_sft/scripts/sokoban/sokoban_grpo.sh

For General Points:

bash

bash debunk_sft/scripts/gp_l/gp_l_grpo.sh