latkes/factprobe-replication-stage1-cooccurrence-v1
factprobe-replication-stage1-cooccurrence-v1 How many documents of the OLMo-2 pretraining corpus contain BOTH names of a probed pair. Computed over all 1,117 token files (15.50 TB, 3.875 trillion tokens) for the 2,172,383 name pairs the model was probed about; 313,576 of them share at least one document. Replaces an earlier version measured on the 192 GB mid-training mix alone, where 80-92% of pairs never co-occurred and the quantity behaved as a yes/no flag rather than a graded… See the full description on the dataset page: https://huggingface.co/datasets/latkes/factprobe-replication-stage1-cooccurrence-v1.
factprobe-replication-stage1-cooccurrence-v1
How many documents of the OLMo-2 pretraining corpus contain BOTH names of a probed pair. Computed over all 1,117 token files (15.50 TB, 3.875 trillion tokens) for the 2,172,383 name pairs the model was probed about; 313,576 of them share at least one document. Replaces an earlier version measured on the 192 GB mid-training mix alone, where 80-92% of pairs never co-occurred and the quantity behaved as a yes/no flag rather than a graded measure.
Dataset Info
- Rows: 313576
- Columns: 3
Columns
Generation Parameters
{
"script_name": "cooccurrence_shard.py",
"model": "OLMo-2 pretraining corpus (not a model run)",
"description": "How many documents of the OLMo-2 pretraining corpus contain BOTH names of a probed pair. Computed over all 1,117 token files (15.50 TB, 3.875 trillion tokens) for the 2,172,383 name pairs the model was probed about; 313,576 of them share at least one document. Replaces an earlier version measured on the 192 GB mid-training mix alone, where 80-92% of pairs never co-occurred and the quantity behaved as a yes/no flag rather than a graded measure.",
"experiment_name": "factprobe-replication",
"job_id": "mll:72741",
"cluster": "mll",
"artifact_status": "final",
"canary": false,
"hyperparameters": {
"source": "document index written during counting, 437 GB over 1,117 shards",
"name_matching": "OLMo token sequences, AI2 splitting rule, word boundaries both ends, both written forms",
"pairs_considered": 2172383
},
"input_datasets": [
"olmo-mix-1124 stage-1 pretraining corpus"
]
}Usage
from datasets import load_dataset
dataset = load_dataset("latkes/factprobe-replication-stage1-cooccurrence-v1", split="train")
print(f"Loaded {len(dataset)} rows")