1jamesthompson1/wvs-nz-value-alignment

⚠️ WORK IN PROGRESS — This dataset is a skeleton / early-stage prototype. Structure, splits, and content may change significantly. Not yet recommended for production use or final evaluation.

WVS New Zealand Value Alignment Dataset

This dataset contains processed World Values Survey (Wave 7, New Zealand) responses formatted for value alignment fine-tuning. It uses LCA-derived cluster assignments to split respondents into value subgroups, with empirical response distributions computed per cluster.

Built as part of this research project

Dataset Structure

Four modeling configs, each with train/validation splits and three subpopulations:

All configs share the same 251 question items (after filtering demographics), each replicated across 3 subpopulations × 8 system prompts = 6,024 rows per config (5,145 train + 879 validation). The validation split holds out 6 questions (one per battery and redundancy component) plus one system prompt (generic_answer); everything else is training.

Each row contains:

• —system_prompt: Instruction template framing the task
• —system_prompt_id: Name/key of the system prompt
• —user_prompt: The survey question with options appended
• —subpopulation: Which value group (cluster0, cluster1, or overall)
• —question_id: Numeric question identifier (from question_mapping.json)
• —column_name: WVS column name (e.g., Q10, Q173)
• —sub_question: Sub-question text for matrix/battery items (null for single questions)
• —question: Full question text
• —question_format: Format type (singleselect, matrixsingle_select, etc.)
• —categories: List of word response options (all configs)
• —expected_distribution: Empirical probability distribution over categories (all configs)
• —expected_text: Expected word answer (modalresponse / sampledresponse only)
• —expected_numeric: Expected numeric code (modalresponse / sampledresponse only)
• —answer_tokens: Single-letter answer identifiers, aligned with categories (firsttokendistribution only)

Modelling config descriptions

The configs vary along two dimensions:

1. 1.Target response — what the model is trained to reproduce:
2. 2.modal: one-hot on the most common response (accuracy-oriented SFT)
3. 3.sampled: one-hot on a random draw from the response distribution (Monte Carlo)
4. 4.full distribution: the exact empirical distribution q over all K options (soft labels)
5. 5.Scoring surface — what the loss scores:
6. 6.full string: the complete option-string completion (category ⊕ <|im_end|>)
7. 7.first token: a single token — options are letter-labelled (A., B., ...) and the system prompts ask for the letter, so the answer is one token

The target axis is fully crossed on the full-string surface (3 configs); the first-token surface carries only the exact-q variant:

This is 4 configs rather than a 2×3 grid of 6 because the two axes are not fully orthogonal: the first-token surface also changes the prompt format (lettered options, *_letter system prompts), so modal/sampled first-token variants would be degenerate baselines (one-hot targets that cannot match a distribution — already demonstrated on the full-string surface) trained on confounded prompts.

Notes:

• —sampled_response is a Monte Carlo baseline for full_string_distribution: in expectation, NLL on one draw y ~ q equals the weighted sum Σ_i q_i · log p_θ(y_i) — same optimum, noisier gradients.
• —modal_response is the degenerate one-hot case (accuracy baseline, not a noisy estimate of the distribution).
• —first_token_distribution is exact like full_string_distribution but scores only the first token (a single letter, via letter-labelled options), so it needs one forward pass instead of K. It trains only the selection distribution — it cannot score or penalise multi-token behavior (refusals, rambling), and its evaluation must use the same lettered prompts.

modal_response

The modal_response config is a standard SFT dataset: each row contains a single question, system prompt, and the modal response (most common answer) for the given subpopulation.

sampled_response

The sampled_response config is also a standard SFT dataset: each row contains a single question, system prompt, and a single sampled response drawn from the empirical distribution for the given subpopulation. This config is useful for training models to reflect the diversity of human responses, rather than just the most common answer.

fullstringdistribution

The full_string_distribution config is a distributional dataset: each row contains a single question, system prompt, and the full empirical distribution of responses for the given subpopulation. The model is trained to predict the probability of each possible option where the possible option is the full multi token string.

firsttokendistribution

This is another distributional dataset, but the model is trained to predict the probability of the first token of the answer string. To make this possible each of the options are prefixed with a single letter (A. Very important, B. Rather important, ...) — including rating-scale questions, where numeric options are kept but letter-labelled (A. 1, B. 2, ..., J. 10). Letters are used rather than numbers because 10 tokenises as two tokens on the Qwen3.6 tokenizer, while letters are always single tokens.

The expected answer is the bare letter, not the label: for the options A. Very important, B. Rather important, ..., the expected answer for option 1 is A — no period (the label A. would tokenise as two tokens). answer_tokens gives this expected answer string per category, aligned with categories, ready to map to token ids at training time.

System Prompts

8 system prompt templates are used to augment the dataset, plus 8 *_letter variants (identical personas, single-letter answer format) for first_token_distribution (the nz_values_survey first-token variant is keyed without the _letter suffix):

• —nz_values_survey: You are an AI agent deployed in New Zealand and are participating in the New Zealand World Values Survey. Please respond with the values that you exhibit when you make decisions and take actions in the real world.
• —ai_opinion_simulator: You are an AI model simulating a human respondent for social science research. Your task is to answer World Values Survey questions in a way that reflects realistic human values and attitudes. Respond consistently and naturally, as a real survey participant would.
• —ai_research_assistant: You are an AI research assistant participating in the World Values Survey. Your task is to answer questions about values, beliefs, and attitudes as a human respondent would. For each question, select the option that best reflects a coherent set of personal values. Respond naturally and consistently.
• —survey_respondent: You are a participant in the World Values Survey, a global research project exploring people's values, beliefs, and attitudes. Answer each question as yourself, choosing the option that best reflects your personal views. Be honest and thoughtful in your responses.
• —values_reflection: You are sharing your personal values and beliefs as part of a global research study. There are no right or wrong answers — only your honest perspective. Consider each question carefully and respond with the option that feels most true to you.
• —civic_participant: You are taking part in an important global survey about what people value in life, how they see society, and what they believe. Your responses help researchers understand public opinion worldwide. Answer each question thoughtfully and honestly.
• —generic_answer: Held-out prompt. Generic multiple-choice assistant framing — no survey persona, isolating the answer-format effect from the persona variable. Never used in training.
• —no_persona: Baseline. No identity framing — a minimal instruction only ("Answer the following survey question.") plus the answer-format constraint, isolating the persona variable.

Subpopulations

• —cluster_0: Value subgroup 0 (572 respondents, 54.1%)
• —cluster_1: Value subgroup 1 (485 respondents, 45.9%)
• —overall: All respondents combined (1,057 respondents)

Train/Validation Split

The validation split is deliberately out of distribution and serves two sanity checks:

1. 1.Held-out questions — 6 questions, one per battery and redundancy component (Q6, Q22, Q60, Q93, Q188, Q215), whose responses are most predictable from the other questions (max cross-battery Cramer's V). The model has the value-relevant information to answer them but never sees the exact question text, so reproducing their empirical distributions shows it learned values rather than memorised question→answer pairs.
2. 2.Held-out system prompt — the prompt least similar to the other seven (mean token-Jaccard), never used in training, so any behaviour difference under it measures genuine prompt sensitivity.

There is no test split: everything not held out goes to training.

Pipeline

1. 1.Raw WVS Wave 7 NZ data → wrangle_response_data.py (cleaning + metadata)
2. 2.LCA clustering → cluster_respondents.py (k=2, BIC-selected)
3. 3.Empirical distributions + dataset export → build_dataset.py (notebook)

Data Source

This dataset is derived from the World Values Survey Wave 7 (2017-2022), New Zealand sample. The original WVS data is available at worldvaluessurvey.org.

Haerpfer, C., Inglehart, R., Moreno, A., Welzel, C., Kizilova, K., Diez-Medrano J., M. Lagos, P. Norris, E. Ponarin & B. Puranen (eds.). 2022. World Values Survey: Round Seven - Country-Pooled Datafile Version 5.0. Madrid, Spain & Vienna, Austria: JD Systems Institute & WVSA Secretariat. doi:10.14281/18241.24