Datasetpublic

CoffeeGitta/difficulty-gsm8k-generations

Generations Dataset: gsm8k LLM-generated solutions across train/validation/test splits for multiple models. Columns Column Type Description problem str Problem statement generated_solutions list Generated solutions with scores success_rate float Fraction of correct generations majority_vote_is_correct int (0/1) Whether majority vote is correct k int Number of samples generated temperature float Sampling temperature max_len int Maximum… See the full description on the dataset page: https://huggingface.co/datasets/CoffeeGitta/difficulty-gsm8k-generations.

sourceHugging Faceupdated 4mo agoView on Hugging Face

0likes73downloads

Dataset Card

Generations Dataset: gsm8k

LLM-generated solutions across train/validation/test splits for multiple models.

Columns

Column	Type	Description
`problem`	str	Problem statement
`generated_solutions`	list	Generated solutions with scores
`success_rate`	float	Fraction of correct generations
`majority_vote_is_correct`	int (0/1)	Whether majority vote is correct
`k`	int	Number of samples generated
`temperature`	float	Sampling temperature
`max_len`	int	Maximum generation length
`model_name`	str	Model used for generation

Usage

python

from datasets import load_dataset

dataset = load_dataset("CoffeeGitta/difficulty-gsm8k-generations", name="<org--model>")
train = dataset["train"]

Citation

bibtex

@article{lugoloobi_llms_2026,
    title = {LLMs Encode Their Failures: Predicting Success from Pre-Generation Activations},
    url = {http://arxiv.org/abs/2602.09924},
    author = {Lugoloobi, William and Foster, Thomas and Bankes, William and Russell, Chris},
    year = {2026},
}