public-records-research/epstractor-raw

Epstractor: Epstein Archives Dataset

A comprehensive archive of documents, images, audio, and video files from multiple Epstein-related releases, including estate records and Department of Justice materials obtained through FOIA requests.

Dataset Description

This dataset contains 59,420 files totaling 115.23 GB from three major document releases, plus 2 large videos (40GB) available via a separate config:

• —Epstein Estate 2025-09: 5 files, 0.09 GB
• —Epstein Estate 2025-11: 26,035 files, 36.56 GB
• —House DOJ 2025-09: 33,380 files, 78.58 GB
• —Large Videos: 2 videos (>2GB each), 40 GB - separate config

File Types

• —Images: ~56,418 files (JPG, TIF) - scanned documents and photographs
• —Text: ~2,897 TXT files - OCR'd and extracted text
• —Audio: 56 WAV files
• —Video: 28 MP4/MOV files
• —Documents: 14 PDF/XLS/XLSX files
• —Metadata: JSON manifests

Directory Structure

The dataset is organized by source for flexible loading:

data/
├── epstein_estate_2025_09/
│   └── epstein_estate_2025_09-00000-of-00001.parquet (1 shard, 0.09 GB)
├── epstein_estate_2025_11/
│   ├── epstein_estate_2025_11-00000-of-00075.parquet
│   ├── epstein_estate_2025_11-00001-of-00075.parquet
│   └── ... (75 shards, 36.56 GB)
├── house_doj_2025_09/
│   ├── house_doj_2025_09-00000-of-00060.parquet
│   ├── house_doj_2025_09-00001-of-00060.parquet
│   └── ... (60 shards, 78.58 GB)
└── large_files/
    └── large_files.parquet (metadata for 2 large videos)

large_files/
├── DOJ-OGR-00022168.MP4 (20 GB)
└── DOJ-OGR-00022169.MP4 (20 GB)

Data Structure

The dataset is stored in Parquet format with the following schema:

Source and Provenance

These materials were officially released by the U.S. House Committee on Oversight and Accountability via public Google Drive folders, making them freely accessible to the public without any access restrictions, paywalls, or distribution limitations.

Official Release Channels:

• —House Oversight Committee public releases (September 2025) via Google Drive
• —Epstein Estate public releases (September & November 2025) via Google Drive
• —Department of Justice FOIA releases (2025)

All files were downloaded from publicly accessible sources with clear governmental intent to disseminate to the public. No proprietary or restricted materials are included.

Chain of Custody:

1. 1.Congressional/Government agencies release materials publicly
2. 2.Materials hosted on open Google Drive folders (no authentication required)
3. 3.Downloaded and aggregated for research accessibility
4. 4.Converted to Parquet format for efficient access
5. 5.Published on HuggingFace for public research use

This is not scraped, leaked, or proprietary content - all materials were intentionally made public by official government sources.

Usage

The dataset is organized by source as separate configs, allowing you to load all data or specific sources:

from datasets import load_dataset

# Load ALL sources as a DatasetDict (136 shards, 59,420 files)
dataset = load_dataset("public-records-research/epstractor-raw")
# Returns: {"epstein_estate_2025_09": Dataset, "epstein_estate_2025_11": Dataset, "house_doj_2025_09": Dataset}

# Access a specific source
estate_2025_11 = dataset["epstein_estate_2025_11"]
house_doj = dataset["house_doj_2025_09"]

# OR load ONLY a specific source directly
estate_2025_11 = load_dataset("public-records-research/epstractor-raw", name="epstein_estate_2025_11")
house_doj = load_dataset("public-records-research/epstractor-raw", name="house_doj_2025_09")

# Access a single file
row = estate_2025_11[0]
print(f"Path: {row['path']}")
print(f"Source: {row['source']}")
print(f"Type: {row['file_type']}")
print(f"Size: {row['file_size']} bytes")

# Save file content to disk
if row['content_available']:
    from pathlib import Path
    file_path = Path(row['source']) / row['path']
    file_path.parent.mkdir(parents=True, exist_ok=True)
    with open(file_path, 'wb') as f:
        f.write(row['content'])

# Filter by file type
images = estate_2025_11.filter(lambda x: x['file_type'] == 'image')
print(f"Total images: {len(images)}")

# Combine all sources if needed
from datasets import concatenate_datasets
combined = concatenate_datasets([
    dataset["epstein_estate_2025_09"],
    dataset["epstein_estate_2025_11"],
    dataset["house_doj_2025_09"]
])

Large Videos

Two video files exceed PyArrow's 2GB binary limit and are provided via a separate large_files config:

from datasets import load_dataset
from huggingface_hub import hf_hub_download

# Load the large_files config
videos = load_dataset("public-records-research/epstractor-raw", name="large_files")

# List available videos
for video in videos['train']:
    print(f"{video['file_id']}: {video['file_size'] / 1_073_741_824:.1f} GB")
    print(f"  Path: {video['path']}")
    print(f"  Repo path: {video['repo_path']}")

# Download a specific video
video = videos['train'][0]
local_path = hf_hub_download(
    repo_id="public-records-research/epstractor-raw",
    repo_type="dataset",
    filename=video['repo_path']
)
# Now use local_path with your video player or processing library

large_files Schema

Dataset Creation

Source Data

This dataset aggregates files from:

1. 1.Epstein Estate release (September 2025)
2. 2.Epstein Estate release (November 2025)
3. 3.House Committee on Oversight and Accountability - DOJ release (September 2025)

All materials were obtained through official channels including FOIA requests and public releases by government entities.

Data Processing

Files were converted to Parquet format using PyArrow with:

• —500 MB maximum shard size (136 total shards)
• —Snappy compression
• —Binary content preservation
• —File metadata extraction

Large Files: 2 video files exceeding 2GB (PyArrow's binary value limit) are available via the separate large_files config. See the "Large Videos" section below for details.

Considerations

• —Size: 115+ GB total - ensure adequate storage and bandwidth
• —Content: Contains legal documents, personal correspondence, and media
• —Large Files: 2 files >2GB have metadata only (content not embedded)
• —Privacy: Materials are from public releases; review applicable privacy considerations

License

These materials are considered public domain as they are works of the U.S. Government or were publicly released through official channels. See USA.gov Government Works for more information.

Individual documents may be subject to additional restrictions or copyright claims. Users should review applicable laws and regulations for their jurisdiction.

Citation

@dataset{epstractor2025,
  title={Epstractor: Epstein Archives Dataset},
  author={Various Government Sources},
  year={2025},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/datasets/public-records-research/epstractor-raw}},
  note={Aggregated from Epstein Estate releases and DOJ FOIA materials}
}

Maintenance

Dataset created: 2025-11-16 Last updated: 2025-11-16 Version: 1.0.0