Hinted
Datasets
All datasets matching “Hinted”hintedselfteacher-nemotron-math-v2-AoPS
hintedselfteacher-nemotron-math-v2-AoPS
This dataset contains a training-ready hinted self-teacher split derived from the AoPS split of nvidia/Nemotron-Math-v2.
The source problems were filtered to the AoPS split with the medium/notool solve rate between 2 and 6. Hints were generated with GPT-5.5 medium using an h17_nt hint-generation prompt. This hint type was close to the best hint type found after doing hint mutations, based on qualitative analysis of token-level hinted… See the full description on the dataset page: https://huggingface.co/datasets/ar0cket1/hintedselfteacher-nemotron-math-v2-AoPS.cot-oracle-eval-hinted-mcq
CoT Oracle Eval: hinted_mcq
GSM8K problems as 4-choice MCQ with hints. 50/50 right/wrong hints, varying subtlety. Source: openai/gsm8k test.
Part of the CoT Oracle Evals collection.
Schema
Field
Description
eval_name
Eval identifier
example_id
Unique example ID
clean_prompt
Prompt without nudge/manipulation
test_prompt
Prompt with nudge/manipulation
correct_answer
Ground truth answer
nudge_answer
Answer the nudge pushes toward… See the full description on the dataset page: https://huggingface.co/datasets/ceselder/cot-oracle-eval-hinted-mcq.hinted_mbpp_llama2_7B_chat
Dataset Card for "hinted_mbpp_llama2_7B_chat"
More Information needed
mmlu_hinted_questions
MMLU Hinted Questions
Dataset Description
This dataset contains multiple-choice questions derived from MMLU and augmented with misleading hints. The misleading hints are intentionally designed to point to an incorrect answer.
The dataset was developed as part of the UnfaithRL project, which studies cue-following and unfaithful reasoning under reinforcement learning with verifiable rewards.
Specifically, it was used to investigate whether language models follow… See the full description on the dataset page: https://huggingface.co/datasets/UnfaithRL/mmlu_hinted_questions.mmlu_hinted_huggingfaceThis is a massive multitask test consisting of multiple-choice questions from various branches of knowledge, covering 57 tasks including elementary mathematics, US history, computer science, law, and more.mmlu_hinted_rollouts
MMLU-with-hint faithfulness eval — flipped-to-hint rollouts (+ judge verdicts)
Companion data for the blog post on side effects of CoT length penalties in RL
(MATS sprint project). Model checkpoints:
brikdavies/RL-length-penalty-checkpoints.
Each row is one MMLU question (~5k question eval, hint placed mid-prompt) where the
model flipped its answer to the hinted answer (unhinted_answer != hinted_answer
and the hinted run's extracted answer equals the hint). Rows carry:
the… See the full description on the dataset page: https://huggingface.co/datasets/lucferon/mmlu_hinted_rollouts.