Datasetpublic

jbduran/vintage-core

Vintage CORE Vintage CORE is a period-aligned version of the DataComp-LM CORE benchmark for language models with a 1930 knowledge cutoff. This repository is the versioned dataset distribution for the johnny0595/vintage-core project. Repository roles Location Role GitHub Code, documentation, tests, evaluator, Colab notebook, and offline data mirror Hugging Face Canonical versioned dataset download The v1.0.0 data payload is identical in both… See the full description on the dataset page: https://huggingface.co/datasets/jbduran/vintage-core.

sourceHugging Faceotherupdated 1mo agoView on Hugging Face

1likes35downloads

Dataset Card

Vintage CORE

Vintage CORE is a period-aligned version of the DataComp-LM CORE benchmark for language models with a 1930 knowledge cutoff. This repository is the versioned dataset distribution for the `johnny0595/vintage-core` project.

Repository roles

Location	Role
GitHub	Code, documentation, tests, evaluator, Colab notebook, and offline data mirror
Hugging Face	Canonical versioned dataset download

The v1.0.0 data payload is identical in both locations. GitHub's DATA_MANIFEST.sha256 and this repository's SHA256SUMS contain the same 44 hashes; paths here omit GitHub's leading data/ directory.

Bundles

—filtered/: 20 task labels and 47,092 unique physical rows after temporal filtering and reviewed small-task backfills.
—restyled/: the same task configuration, items, row order, choices, answers, and protected continuations, with eligible prose conservatively restyled toward an 1800–1930 register.

Original CORE is not mirrored. The evaluator retrieves it from nanochat's published bundle URL. HellaSwag and HellaSwag zero-shot intentionally share one physical file, so each directory contains 19 JSONL files for 20 evaluations.

Download

python

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="jbduran/vintage-core",
    repo_type="dataset",
    revision="v1.0.0",
    allow_patterns=["filtered/**", "restyled/**", "SHA256SUMS"],
    local_dir="vintage-core-v1.0.0",
)

Pin revision="v1.0.0" when reproducing the published experiment. main carries the latest dataset-card metadata while retaining the same v1.0.0 payload.

Validate and evaluate

bash

git clone https://github.com/johnny0595/vintage-core.git
cd vintage-core
uv sync --group dev
uv run vintage-core-validate

For checkpoint evaluation, install the eval extra and follow the GitHub run guide. The repository also includes a GPU-ready Colab notebook.

Recorded results

Common-20 is the centered CORE mean over the 20 task labels shared by Original, Filtered, and Restyled CORE.

Model	Original	Filtered	Restyled
Think Unbounded d32	16.3%	17.5%	17.6%
Modern nanochat d24	26.3%	26.2%	25.3%
GPT-1900 d34	12.1%	12.7%	13.9%

These are recorded full-benchmark results, not measurements produced by this metadata update. See the final report for protocol, construction details, costs, limitations, and source records.

License and provenance

The companion repository's MIT License applies to project code and original documentation. It does not relicense the underlying benchmark text. The bundles adapt third-party evaluation datasets with source-specific terms, and some incorporated tasks do not have a blanket redistribution grant documented by this project. Public availability is not a representation that all dataset content is MIT-licensed.

Review the project's data provenance and NOTICE before use or redistribution. Please cite DataComp-LM and the original evaluation datasets when publishing results.

Vintage CORE was developed by Jonathan Duran-Ortiz at Unbounded Labs.