spadeMIA/GoodWiki_Corpus_1024_2040

GoodWiki 1024–2040: paragraph-truncated MIA fine-tuning corpus

A deterministic, paragraph-truncated corpus of English Wikipedia Good/Featured articles, built from `euirim/goodwiki` for membership inference attack (MIA) experiments on fine-tuned language models.

Membership labels are defined relative to the fine-tuning population. train (10,000 rows) is the only split used for fine-tuning, and every row has label = 1. test (1,000 rows) remains held out, and every row has label = 0. evaluation (700 rows) is a fixed, balanced MIA candidate set whose existing labels are retained. Fine-tune on train only. Never use test or evaluation for gradient updates, early stopping, checkpoint selection, or attack hyperparameter tuning.

Project and team

This corpus was built for a membership inference attack research project at the Security, Privacy and Data Engineering (SPADE) Lab, Koç University.

Schema

All three splits use the same two-column schema:

label is dataset metadata for MIA evaluation. It is not the token-level causal-LM labels tensor and must not be passed through as a language-model training target.

Dataset layout and loading

This release has one default Hugging Face dataset configuration and three splits. The dataset card intentionally declares no configs: stanza, so callers do not select a configuration or subset.

Load any split without a configuration argument:

from datasets import load_dataset

repo_id = "spadeMIA/GoodWiki_Corpus_1024_2040"
train = load_dataset(repo_id, split="train")
test = load_dataset(repo_id, split="test")
evaluation = load_dataset(repo_id, split="evaluation")

Evaluation-set semantics

The existing evaluation Parquet file is retained unchanged. Its 350 member candidates have label = 1. Its 350 non-member candidates have label = 0 and are independent, same-source GoodWiki records, not rows sampled from test. For this benchmark, a non-member label means that the candidate record was not in the train fine-tuning population. It does not mean that the record came from test.

The shared schema is a storage and loading convention, not a change to the text corpus. The train and test migrations add only their constant scalar label columns. Their text values, row counts, and row order are immutable. The existing evaluation Parquet file, including its text values, labels, row order, and row count, is immutable as well.

Token contract

All counts use the EleutherAI/pythia-2.8b tokenizer (shared across the Pythia suite) with add_special_tokens=False. Training-time sequence construction appends exactly one EOS token (id 0), so:

1,024 <= stored-text tokens <= 2,039
stored-text tokens + 1 EOS  <= 2,040

The 2,039 cap is not arbitrary: it is the 2,040-token training-sequence budget minus the one reserved EOS. The EOS is not stored in text.

How it was built

1. 1.Source: euirim/goodwiki (snapshot fingerprint 458c4deaf17608e7), split train, column markdown — 44,754 articles.
2. 2.Minimal normalization only: \r\n/\r → \n plus one outer strip() per article. No Markdown conversion, no HTML cleaning, no lowercasing, no Unicode normalization; headings, links, tables, lists, and quotes are preserved as-is.
3. 3.Paragraph-boundary truncation: articles over 2,039 tokens are shortened by removing whole paragraph-level blocks (maximal runs of nonblank lines) from the end only — never from the beginning or middle, never splitting a block. Every candidate prefix was re-tokenized exactly before acceptance.
4. 4.Dangling-heading rule: no stored text ends on a standalone ATX heading with no body beneath it.
5. 5.Exclusions: articles below 1,024 tokens (2,459) and articles with no valid complete-block prefix in range (29) were dropped.
6. 6.Exact deduplication of finalized texts (0 found), then a single seed-42 NumPy permutation selected 10,000 train + 1,000 test rows from the 42,266 eligible articles; the remaining 31,266 are unused.

Both classes were processed by the identical pipeline — no member/non-member asymmetry that a text-only classifier could exploit.

Statistics

Processing funnel

70.8% of source articles exceeded the 2,039-token cap; 79.4% of eligible rows had at least one block removed (20.6% kept in full).

Stored-text token counts (re-verified by full re-tokenization)

The two classes are drawn from one seeded permutation of the same pool, so their length distributions match by construction (largest percentile gap: 64 tokens at p5, sampling noise).

Paragraph blocks removed per article

Validation

The original corpus validation re-verified 10,000 train rows and 1,000 test rows: every text re-tokenizes within 1,024–2,039 tokens (+1 EOS ≤ 2,040), no row is empty or has outer whitespace, no text ends at a dangling heading, there are zero duplicates within either split, and there is zero train/test overlap.

The shared-schema release additionally verifies all of the following:

1. 1.train, test, and evaluation each expose exactly text and label, in that order.
2. 2.train has 10,000 rows with only label = 1; test has 1,000 rows with only label = 0; evaluation has 700 rows with 350 labels of each value.
3. 3.The ordered text sequences in train and test are byte-for-byte unchanged from the original corpus. Their existing row counts and row order are unchanged.
4. 4.The evaluation Parquet artifact is unchanged, including its text values, labels, row order, and row count.
5. 5.load_dataset(repo_id, split=split_name) succeeds for all three split names without a configuration argument.

Ordered content hashes (SHA-256 over the length-prefixed text sequence):

train: 9ee9a293c58451cc1ffa05ef630e3ae0da26e4c71a0fbf6ad9541494e9fc3adc
test:  68f88f610108b710b185751468a787807f4f3647eddd0bfecb2bf0563685fcc4

These are text-only invariants. Adding the constant membership column to train or test must not change either value.

Provenance

• —Built 2026-07-23 by Domains/GoodWiki/data/goodwiki_1024_2040_preprocessing.ipynb; reviewed by goodwiki_1024_2040_dataset_review.ipynb (private repository MeldaPaksoy/-LLM-MIA-Partial-Samples lineage / MIA-Research-Lab).
• —Seed 42 throughout the original train/test corpus construction; full aggregate build audit in goodwiki_1024_2040_audit.json alongside the notebooks' artifacts.
• —Environment: Python 3.11.14, transformers 5.13.0, tokenizers 0.22.2, datasets 5.0.0, numpy 2.4.6, pandas 3.0.3, pyarrow 24.0.0.

Related corpora — do not mix

This corpus supersedes mia-llm/goodwiki_finetune_corpus (uncapped article records; median ≈3,445 Pythia tokens, ~79% exceeding the 2,048-token Pythia context window) and its sentence-truncated 2,047-token derivative for the paragraph-truncation line of experiments. Checkpoints are only interpretable against the exact corpus they were trained on — train new target models on this corpus; never compare attacks across checkpoints trained on different GoodWiki corpora.

Licensing and attribution

Article text originates from English Wikipedia and is licensed CC BY-SA 4.0; derived text distributed here carries the same license and share-alike obligations. The upstream euirim/goodwiki release declares an MIT license for its dataset packaging and code. This derived corpus preserves article text verbatim (prefix slices only) and stores no titles, page ids, or revision ids; see euirim/goodwiki for per-article Wikipedia provenance.