datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
hintedselfteacher-nemotron-math-v2-AoPS
hintedselfteacher-nemotron-math-v2-AoPS
This dataset contains a training-ready hinted self-teacher split derived from the AoPS split of nvidia/Nemotron-Math-v2.
The source problems were filtered to the AoPS split with the medium/notool solve rate between 2 and 6. Hints were generated with GPT-5.5 medium using an h17_nt hint-generation prompt. This hint type was close to the best hint type found after doing hint mutations, based on qualitative analysis of token-level hinted… See the full description on the dataset page: https://huggingface.co/datasets/ar0cket1/hintedselfteacher-nemotron-math-v2-AoPS.cot-oracle-eval-hinted-mcq
CoT Oracle Eval: hinted_mcq
GSM8K problems as 4-choice MCQ with hints. 50/50 right/wrong hints, varying subtlety. Source: openai/gsm8k test.
Part of the CoT Oracle Evals collection.
Schema
Field
Description
eval_name
Eval identifier
example_id
Unique example ID
clean_prompt
Prompt without nudge/manipulation
test_prompt
Prompt with nudge/manipulation
correct_answer
Ground truth answer
nudge_answer
Answer the nudge pushes toward… See the full description on the dataset page: https://huggingface.co/datasets/ceselder/cot-oracle-eval-hinted-mcq.hinted_mbpp_llama2_7B_chat
Dataset Card for "hinted_mbpp_llama2_7B_chat"
More Information needed
mmlu_hinted_questions
MMLU Hinted Questions
Dataset Description
This dataset contains multiple-choice questions derived from MMLU and augmented with misleading hints. The misleading hints are intentionally designed to point to an incorrect answer.
The dataset was developed as part of the UnfaithRL project, which studies cue-following and unfaithful reasoning under reinforcement learning with verifiable rewards.
Specifically, it was used to investigate whether language models follow… See the full description on the dataset page: https://huggingface.co/datasets/UnfaithRL/mmlu_hinted_questions.mmlu_hinted_rollouts
MMLU-with-hint faithfulness eval — flipped-to-hint rollouts (+ judge verdicts)
Companion data for the blog post on side effects of CoT length penalties in RL
(MATS sprint project). Model checkpoints:
brikdavies/RL-length-penalty-checkpoints.
Each row is one MMLU question (~5k question eval, hint placed mid-prompt) where the
model flipped its answer to the hinted answer (unhinted_answer != hinted_answer
and the hinted run's extracted answer equals the hint). Rows carry:
the… See the full description on the dataset page: https://huggingface.co/datasets/lucferon/mmlu_hinted_rollouts.mmlu_hinted_huggingfaceThis is a massive multitask test consisting of multiple-choice questions from various branches of knowledge, covering 57 tasks including elementary mathematics, US history, computer science, law, and more.hinted_mmlu_560This is a massive multitask test consisting of multiple-choice questions from various branches of knowledge, covering 57 tasks including elementary mathematics, US history, computer science, law, and more.cot-oracle-eval-hinted-mcq-truthfulqaeducation_qna_hinted_staticmmlu_1120_hintedThis is a massive multitask test consisting of multiple-choice questions from various branches of knowledge, covering 57 tasks including elementary mathematics, US history, computer science, law, and more.mmlu_1120_hinted_testThis is a massive multitask test consisting of multiple-choice questions from various branches of knowledge, covering 57 tasks including elementary mathematics, US history, computer science, law, and more.education_qna_hintedlmfd__hinted_method__gpt4omini
Dataset card for lmfd__hinted_method__gpt4omini
This dataset was made with Curator.
Dataset details
A sample from the dataset:
{
"question": "What is the solution to the long multiplication equation below?\n\n8274 x 3529\n\nThink step by step.",
"solution": "29198946",
"eval_prompt": "What is the solution to the long multiplication equation below?\n\n8274 x 3529\n\nThink step by step.\n\nBreak this question down step by step using the **distributive… See the full description on the dataset page: https://huggingface.co/datasets/TAUR-dev/lmfd__hinted_method__gpt4omini.test_mmlu_hintedThis dataset contains a copy of the cais/mmlu HF dataset but without the auxiliary_train split that takes a long time to generate again each time when loading multiple subsets of the dataset.
Please visit https://huggingface.co/datasets/cais/mmlu for more information on the MMLU dataset.
evals__lmfd__hinted_method__gpt4omini__samplesconvos__lmfd__hinted_method__gpt4ominilmfd__hinted_method_w_verification__gpt4omini
Dataset card for lmfd__hinted_method_w_verification__gpt4omini
This dataset was made with Curator.
Dataset details
A sample from the dataset:
{
"question": "What is the solution to the long multiplication equation below?\n\n8274 x 3529\n\nThink step by step.",
"solution": "29198946",
"eval_prompt": "What is the solution to the long multiplication equation below?\n\n8274 x 3529\n\nThink step by step.\n\nBreak this question down step by step using the… See the full description on the dataset page: https://huggingface.co/datasets/TAUR-dev/lmfd__hinted_method_w_verification__gpt4omini.education_qna_hinted_qwen05evals__full_TAUR_dev__convos__lmfd__hinted_method__gpt4omini__samplesevals__full_TAUR_dev__convos__lmfd__hinted_method__gpt4omini__resultsevals__lmfd__hinted_method__gpt4omini__resultspostgresql-hinted-jobcebmmlu_560_haiku_hinted