Datasetpublic

mysocratesnote/jfk-files-text

National Archives JFK Files Text Dataset This dataset contains extracted text from the JFK assassination records released by the National Archives. The dataset preserves the original directory structure from archive.gov while providing significant performance and storage benefits for data analysis, AI applications, and large-scale processing. Dataset Structure The dataset is structured with the following columns: Column Description year The release year… See the full description on the dataset page: https://huggingface.co/datasets/mysocratesnote/jfk-files-text.

sourceHugging Facemitupdated 1y agoView on Hugging Face

0likes58downloads

Dataset Card

National Archives JFK Files Text Dataset

This dataset contains extracted text from the JFK assassination records released by the National Archives. The dataset preserves the original directory structure from archive.gov while providing significant performance and storage benefits for data analysis, AI applications, and large-scale processing.

Dataset Structure

The dataset is structured with the following columns:

Column	Description
`year`	The release year of the document (e.g., 2017, 2018, 2021, 2022, 2023, 2025)
`path`	The relative path to the file, mirroring the archives.gov structure (e.g., `/releases/2022`)
`filename`	The original filename (e.g., `docid-32244106.md`)
`record_number`	NARA record number
`nara_release_date`	NARA release date
`formerly_withheld`	NARA formerly withheld reason
`agency`	Agency
`document_date`	Document date
`document_type`	Document type
`file_number`	NARA file number
`to_name`	To name
`from_name`	From name
`title`	Title
`number_of_pages`	Document number of pages
`originator`	Originator
`record_series`	Record series
`review_date`	NARA review date
`comments`	Comments
`pages_released`	Pages released
`content`	The full text content of the document

Path Organization

The path column preserves the original organizational structure:

—/releases - Files from the 2017 release
—/releases/additional - Additional files from the 2017 release
—/releases/2018 - Files from the 2018 release
—/releases/2021 - Files from the 2021 release
—/releases/2022 - Files from the 2022 release
—/releases/2023 - Files from the 2023 release
—/releases/2023/08 - Files from August 2023 subset
—/releases/2025/0318 - Files the 2025 release

Current Status

Release Year	Status	Extraction Method	Files Extracted	Size	Total Files Listed
2025	✅ Complete	Apple Vision OCR	2,566	8.12GB	2,566
2023	✅ Complete	Apple Vision OCR	2,693	6.20GB	2,693
2022	✅ Complete	Apple Vision OCR	13,199	14.15GB	13,199
2021	✅ Complete	Apple Vision OCR	1,484	1.36GB	1,484
2017-2018	✅ Complete	Apple Vision OCR	53,526	57.18GB	53,547

Note: 34 files in the 2022 release and 5 files in the 2021 release tie to multiple record numbers listed in the .xlsx files which have more rows than unique file names (13,263 and 1,491 resptively). The 2017-2018 release xlsx file contains 6 bad links, but the 2017-2018 release website lists two files not included in the xlsx in the /additional path. The 2017-2018 release all contains 19 audio files (17 .wav, 2 .mp3). Transcripts of the two .mp3 files are included. The 17 .wav files are very poor quailty with lots of blank space (they may be added later).

Archive Statistics

—Total archive size: 87 GB
—Total files: 73,485
—Extracted text in markdown format available on GitHub

WebUI

A simple WebUI to query to the entire archive using the DeepSeek R1 Distill Llama 70B LLM is available at https://jfkfiles.app.

Acknowledgments

—The National Archives for providing the JFK assassination records
—Contributors to the various open-source tools used in this project