Datasetpublic

latkes/factprobe-replication-stage2-fact-counts-v1

factprobe-replication-stage2-fact-counts-v1 How often the 7B mid-training corpus states each fact in so many words -- 'Netherlands borders Germany' -- rather than merely naming both entities in one document. 383,132 sentences were searched: four phrasings per relation, both directions, each entity written with its canonical name. 1,305 occur at all, 6,660 occurrences in total. This corpus is 0.19 TiB; the same measurement over the 14 TiB pretraining corpus is being counted now… See the full description on the dataset page: https://huggingface.co/datasets/latkes/factprobe-replication-stage2-fact-counts-v1.

sourceHugging Facemitupdated 1mo agoView on Hugging Face

0likes15downloads

Dataset Card

factprobe-replication-stage2-fact-counts-v1

How often the 7B mid-training corpus states each fact in so many words -- 'Netherlands borders Germany' -- rather than merely naming both entities in one document. 383,132 sentences were searched: four phrasings per relation, both directions, each entity written with its canonical name. 1,305 occur at all, 6,660 occurrences in total. This corpus is 0.19 TiB; the same measurement over the 14 TiB pretraining corpus is being counted now and is the one that matters.

Dataset Info

—Rows: 1305
—Columns: 11

Columns

Column	Type	Description
alias	Value('string')	the sentence searched for
countleadspace	Value('int64')	occurrences of the space-preceded form
count_bare	Value('int64')	occurrences of the bare form, after a line break or quote
count	Value('int64')	the two summed
corpus	Value('string')	which body of training data was counted
construct	Value('string')	the rule that produced the count
relation	Value('string')	P26 spouse, P3373 sibling, P190 twinned town, P47 borders with
subject	Value('string')	Wikidata item of the entity named first in the forward direction
object	Value('string')	Wikidata item of the other entity
phrasing	Value('string')	which of the four wordings for that relation
direction	Value('string')	forward names the subject first; backward names the object first. The point of counting both is to see whether the corpus states a fact one way round more often than the other

Generation Parameters

json

{
  "experiment_name": "factprobe-replication",
  "cluster": "spark",
  "artifact_status": "final",
  "canary": false,
  "script_name": "build_proposition_patterns.py",
  "model": "n/a",
  "description": "How often the 7B mid-training corpus states each fact in so many words -- 'Netherlands borders Germany' -- rather than merely naming both entities in one document. 383,132 sentences were searched: four phrasings per relation, both directions, each entity written with its canonical name. 1,305 occur at all, 6,660 occurrences in total. This corpus is 0.19 TiB; the same measurement over the 14 TiB pretraining corpus is being counted now and is the one that matters.",
  "hyperparameters": {
    "sentences_searched": 383132,
    "sentences_found": 1305
  },
  "input_datasets": [
    "allenai/dolmino-mix-1124",
    "Zenodo 10.5281/zenodo.15092789"
  ]
}

Usage

python

from datasets import load_dataset

dataset = load_dataset("latkes/factprobe-replication-stage2-fact-counts-v1", split="train")
print(f"Loaded {len(dataset)} rows")