Windsao/eis-text250

EIS-Text250: 1970s U.S. Environmental Impact Statements (text-only)

Per-page OCR/extraction text for 250 scanned 1970s U.S. federal Environmental Impact Statements (EIS) from the Northwestern University Library collection — the text-only companion to `Windsao/eis-subset50` (which carries full page images for a 50-doc subset). Built to test how current models handle long, dense, historical government text: mean ~300 pages/doc, 1970s typewriter prose, OCR noise from degraded microfilm.

Files

• —text.jsonl — one row per page, all 250 documents:

{
  "doc":       "p1274_35556039648936",
  "page":      12,
  "model":     "marker",            // extraction pipeline: "marker" or "chandra"
  "text":      "...page text...",
  "image_url": "https://nu-impulse-production.s3.amazonaws.com/..."
}

• —docs.txt — 250 rows, tab-separated: doc_id models pages group (group = subset50 if the doc is also in the image subset, else text-only).

No images are included; image_url points to the publicly readable full-page scan (~2400×3200 JPG) on S3 if you need pixels for a specific page. For 50 of these documents, all page images are packaged in Windsao/eis-subset50.

Selection

250 of the 277 documents with high-quality text extraction in the corpus (marker or chandra pipelines): all 50 docs of eis-subset50, plus the 200 largest remaining docs by page count (the 27 shortest were dropped).

• —marker pages: text flattened from marker's block tree (HTML stripped).
• —chandra pages: raw markdown including model-generated figure alt-text.
• —Pages covered by both pipelines use chandra.

Known issues / gotchas

• —Page 1 of many documents is a microfilm resolution test chart, not content.
• —A small fraction of pages are legitimately empty (blank microfilm frames).
• —OCR quality varies with scan degradation; text may contain OCR errors — treat as noisy ground truth.
• —marker emits no figure descriptions; only chandra docs describe figures.

Provenance & licensing

• —Source documents: U.S. federal government EIS filings (1970s) — works of the U.S. federal government are in the public domain (17 U.S.C. § 105).
• —Scans: Northwestern University Library digitization (Impulse pipeline).
• —OCR/extraction: marker and chandra pipelines over the scans.
• —Compiled by Shang Wu (Northwestern) for research on long-document understanding. Please cite this dataset page if you use it.