Alberto1231/human_study

human_study

Free-form human Reddit responses derived from `snap-stanford/user_study_annotations`. The original Reddit post text comes from the HumanLM authors' `reddit_post_dict_testset.json`.

Each row contains an original Reddit post in prompt and the response written by a human-study participant in target. Model responses, generated personas, comparison judgments, and worker identifiers are intentionally excluded.

Deduplication and splits

• —Source annotation revision: b19978763f7a1e4fcf71c2facfa65af0136cc425
• —Random seed: 42
• —Exactly one annotation is randomly selected per unique post_id.
• —post_id is unique across the complete derived dataset.
• —The 5 most compact selected rows form the few-shot train split to keep prompts within small model context windows.
• —All remaining unique posts form the scored test split.
• —Train rows: 5
• —Test rows: 59

The prompt contract is:

Post:
<original Reddit post>

Response:<SPACE>

The Response: cue ends with exactly one ASCII space so lm-eval can append few-shot targets and begin generation at a natural token boundary. The target is the participant's original free-form response.