CoolFace
Datasetpublic

Eugleo/pretraining-priors-political-partisan

pretraining-priors political partisan answers For every one of the 6,820 IdeoINST questions (Chen et al., EMNLP 2024, arXiv:2402.11725; the prompt set Eugleo/pretraining-priors-political-eval), one claude-sonnet-5 call wrote a very strongly and clearly left-leaning answer and a very strongly and clearly right-leaning answer (about three sentences each, first person, no partisan self-label), and tagged the question with one 0/1 flag per domain. A flag cat_<domain> is 1 when a… See the full description on the dataset page: https://huggingface.co/datasets/Eugleo/pretraining-priors-political-partisan.

sourceHugging Facemitupdated 17d agoView on Hugging Face
0likes73downloads
Dataset Card

pretraining-priors political partisan answers

For every one of the 6,820 IdeoINST questions (Chen et al., EMNLP 2024, arXiv:2402.11725; the prompt set Eugleo/pretraining-priors-political-eval), one claude-sonnet-5 call wrote a very strongly and clearly left-leaning answer and a very strongly and clearly right-leaning answer (about three sentences each, first person, no partisan self-label), and tagged the question with one 0/1 flag per domain. A flag cat_<domain> is 1 when a person very right-leaning in that domain alone, with no preference in any other, would give the right-leaning answer -- a multi-label reading of the question, next to IdeoINST's own single topic.

columnmeaning
id, topic, split, promptas in the prompt set (topic is IdeoINST's folder)
left_answer, right_answerthe two answers
cat_crime_and_gun ... cat_scienceSonnet's domain flags, 0/1
n_categoriesnumber of flags set
reasoningSonnet's one or two sentences on the domains
check_problemsthe generator's offline checks (sentence count, self-labels); empty = fine
left_stance, right_stancethe exp-086 stance judge's five-way verdict on each answer (claude-sonnet-5); null if unjudged

Splits follow the prompt set: test = the last 256 questions of every topic (1,536 rows), train = the rest (5,284 rows). Flag counts over all rows: crimeandgun 1,315, economyandinequality 1,850, genderandsexuality 1,263, immigration 1,148, race 1,267, science 1,135. Judge verdicts present on 13,640 of 13,640 answers.

Built by experiments/exp086_political_stance/generate_partisan_answers.py (prompt sha 0c1eae29a9f0) at commit 479ba5417612dd74dd0f4f63a4a5ee2e4e9aac00 in pretraining-priors; see manifest.json for per-file sha256s.