CoolFace
Datasetpublic

latkes/factprobe-replication-generation-spouse-13b-base-v1

factprobe-replication-generation-spouse-13b-base-v1 Free-form spouse generation: for every P26 subject NAME form, the base model was asked 'Who is the {spouse} of ? Answer with just the name:' with 4 in-context demos, and sampled 5 times with nucleus sampling (top_p=0.95, temperature=1.0, max_tokens=64, stop at newline). 28,815 subject names. Companion to the P(Yes) probing datasets — this is what the model GENERATES, not a yes/no score. Dataset Info Rows: 30796… See the full description on the dataset page: https://huggingface.co/datasets/latkes/factprobe-replication-generation-spouse-13b-base-v1.

sourceHugging Facemitupdated 22d agoView on Hugging Face
0likes43downloads
Dataset Card

factprobe-replication-generation-spouse-13b-base-v1

Free-form spouse generation: for every P26 subject NAME form, the base model was asked 'Who is the {spouse} of <name>? Answer with just the name:' with 4 in-context demos, and sampled 5 times with nucleus sampling (topp=0.95, temperature=1.0, maxtokens=64, stop at newline). 28,815 subject names. Companion to the P(Yes) probing datasets — this is what the model GENERATES, not a yes/no score.

Dataset Info

  • —Rows: 30796
  • —Columns: 13

Columns

ColumnTypeDescription
relationValue('string')P26 (spouse).
subjectValue('string')subject entity QID.
alias_sValue('string')the subject NAME string put in the prompt (surface-form level; every alias probed).
modelValue('string')HF model id.
revisionValue('string')HF revision (main = base).
top_pValue('float64')nucleus sampling top_p.
temperatureValue('float64')sampling temperature.
nValue('int64')samples per subject name.
max_tokensValue('int64')generation cap.
seedValue('int64')No description provided
prompt_hashValue('string')sha1[:12] of the exact prompt.
demo_leakValue('bool')No description provided
samplesList(Value('string'))the 5 generated answers (free-form names), one nucleus sample each; full text, no truncation.

Generation Parameters

json
{
  "script_name": "generate_spouse.py",
  "model": "allenai/OLMo-2-1124-13B @ main (base, end of stage-2 mid-training)",
  "description": "Free-form spouse generation: for every P26 subject NAME form, the base model was asked 'Who is the {spouse} of <name>? Answer with just the name:' with 4 in-context demos, and sampled 5 times with nucleus sampling (top_p=0.95, temperature=1.0, max_tokens=64, stop at newline). 28,815 subject names. Companion to the P(Yes) probing datasets \u2014 this is what the model GENERATES, not a yes/no score.",
  "hyperparameters": {
    "top_p": 0.95,
    "temperature": 1.0,
    "n_samples": 5,
    "max_tokens": 64,
    "relation": "P26"
  },
  "experiment_name": "factprobe-replication",
  "job_id": "mll:78619 (7b) / mll:80336 (13b)",
  "cluster": "mll",
  "artifact_status": "final",
  "canary": false,
  "input_datasets": []
}

Usage

python
from datasets import load_dataset

dataset = load_dataset("latkes/factprobe-replication-generation-spouse-13b-base-v1", split="train")
print(f"Loaded {len(dataset)} rows")