ai-historian/german-newspaper-pages
📋 On the licensing of this data Every item here is labelled to the best of my ability. The rights statement is taken per issue from what the holding institution recorded in the Deutsche Digitale Bibliothek, captured at download time — never inferred, and never applied at newspaper level to issues that may differ. Labelling at this scale is imperfect. If you believe anything here is mislabelled, or you hold rights in any of this material, please write to lorenz.hufe@posteo.de —… See the full description on the dataset page: https://huggingface.co/datasets/ai-historian/german-newspaper-pages.
Licensing notice and contact for rights queries
Acknowledge the 15 holding institutions
Add content notice on NS-era material (research use only)
Open the dataset mid-corpus (1880) instead of at an extreme
Sort index rows newest-first
List shards newest-first so the preview starts at 1964
Document the chronological shard layout
Per-shard date ranges (2,523 shards)
Single webdataset config; index published as its own dataset
Re-resolve index config data files
Declare default (webdataset) and index (parquet) configs
Add Parquet metadata index (7,000,060 rows) as the 'index' config
Dataset card: measured figures
add 40 shards (pages-00062.tar..pages-00398.tar)
add 40 shards (pages-02161.tar..pages-02317.tar)
add 40 shards (pages-02320.tar..pages-02482.tar)
add 40 shards (pages-01999.tar..pages-02155.tar)
add 40 shards (pages-01825.tar..pages-01996.tar)
add 40 shards (pages-01322.tar..pages-01541.tar)
add 40 shards (pages-01037.tar..pages-01211.tar)
add 40 shards (pages-01214.tar..pages-01520.tar)
add 40 shards (pages-02169.tar..pages-02258.tar)
add 1 shards (pages-02522.tar..pages-02522.tar)
add 3 shards (pages-01547.tar..pages-01553.tar)
add 10 shards (pages-02485.tar..pages-02521.tar)
add 40 shards (pages-01489.tar..pages-01651.tar)
add 40 shards (pages-02350.tar..pages-02439.tar)
add 40 shards (pages-02259.tar..pages-02349.tar)
add 40 shards (pages-02440.tar..pages-02520.tar)
add 40 shards (pages-01917.tar..pages-02168.tar)
add 40 shards (pages-01144.tar..pages-01315.tar)
add 40 shards (pages-01068.tar..pages-01503.tar)
add 40 shards (pages-01489.tar..pages-01651.tar)
add 40 shards (pages-01144.tar..pages-01315.tar)
add 40 shards (pages-01506.tar..pages-01914.tar)
add 40 shards (pages-01657.tar..pages-01822.tar)
add 40 shards (pages-00606.tar..pages-01050.tar)
add 40 shards (pages-00063.tar..pages-00594.tar)
add 40 shards (pages-01321.tar..pages-01483.tar)
add 40 shards (pages-02033.tar..pages-02243.tar)
add 40 shards (pages-00961.tar..pages-01141.tar)
add 40 shards (pages-02249.tar..pages-02405.tar)
add 40 shards (pages-01559.tar..pages-01727.tar)
add 40 shards (pages-01733.tar..pages-02030.tar)
add 40 shards (pages-01781.tar..pages-02171.tar)
add 27 shards (pages-02411.tar..pages-02519.tar)
add 40 shards (pages-00638.tar..pages-00854.tar)
add 40 shards (pages-00857.tar..pages-01034.tar)
add 40 shards (pages-00401.tar..pages-00635.tar)
add 40 shards (pages-00595.tar..pages-00781.tar)