intellekthq/enron-ferc-pst

Enron FERC email corpus in native PST

The EDRM Enron v2 email corpus in Microsoft PST format, modified to reduce personal privacy risk. Mailbox structure, MAPI metadata, message bodies, and retained attachments are preserved.

The release contains 171 PST files in data/, with one or more files per custodian.

Possible uses include email research, e-discovery testing, information retrieval, and agentic research benchmarks.

Querying the corpus

The `duckdb-pst` extension reads PST files directly.

SELECT count(*)
FROM read_pst_messages('data/*.pst');

Read complete bodies for a small result set:

SELECT pst_path, node_id, message_delivery_time, sender_name, subject, body
FROM read_pst_messages('data/*.pst', read_body_size_bytes = 0)
WHERE subject ILIKE '%California%'
LIMIT 20;

Attachment bytes are opt-in:

SELECT m.pst_path,
       m.node_id AS message_node_id,
       a.node_id AS attachment_node_id,
       a.filename,
       a.bytes
FROM read_pst_messages('data/*.pst', read_attachment_body = true) AS m,
     UNNEST(m.attachments) AS t(a)
WHERE a.filename ILIKE '%.pdf'
LIMIT 10;

Message bodies are capped at 1 MB by default. Set read_body_size_bytes = 0 to read complete bodies, and enable read_attachment_body only when attachment bytes are needed.

A node ID is unique only within a PST. Identify a record by (pst_path, node_id).

Agent skill

This dataset includes a duckdb-pst skill for agents that support the SKILL.md format. From the dataset root, link it into the appropriate skill directory:

# Codex
mkdir -p ~/.codex/skills
ln -s "$PWD/skills/duckdb-pst" ~/.codex/skills/duckdb-pst

# Claude Code
mkdir -p ~/.claude/skills
ln -s "$PWD/skills/duckdb-pst" ~/.claude/skills/duckdb-pst

Privacy modifications

The source corpus contained 1,227,193 messages and 492,979 attachments. This release removes 1,015 messages (0.08%) and 39,147 attachments (7.94%). Of the removed attachments, 39,051 were classified deletions and 96 were removed with their parent messages.

The review focused on personal privacy rather than Enron's former corporate interests. Ordinary business correspondence, spam, newsletters, office contact information, and non-residential addresses were retained. Routine health and family references were retained unless they disclosed serious or stigmatised conditions, intimate matters, or information about a minor covered by the removal policy.

The policy also retains obsolete credentials and expired payment-card data where the review found no material current personal risk. This is not a general security or confidentiality scrub.

Attachments without extracted text

The largest removal category comprises 34,554 attachments for which the original scrub obtained no text. Most are small images, media files, fonts, or link wrappers; others could not be parsed or exceeded the OCR time limit. These files were removed conservatively and should not be interpreted as 34,554 confirmed privacy findings.

Review and limitations

Complete message bodies and extracted attachment text were checked with pattern, heuristic, and semantic review methods. Candidate records were adjudicated against a written privacy policy, affected PST free space was overwritten, and all 171 release PST files were rescanned.

All findings produced by the configured release checks were removed or explicitly retained under policy. This does not establish that the corpus contains no personal or sensitive information.

• —Material that does not match the review methods may remain.
• —Some attachments could not be parsed and were removed without a content-based privacy finding.
• —Three residential addresses found during a widened review remain retained under policy.
• —Other projects may choose a stricter boundary for obsolete credentials or expired payment-card data.

Treat this release as a risk-reduced research corpus, not as a guarantee of complete de-identification.

If you find material that should be reviewed, open a dataset discussion with the PST filename and node ID. Do not repost the sensitive text. Material changes are recorded in CHANGELOG.md.

Provenance and license

The embedded corpus notice identifies ZL Technologies, Inc. as the producer of the EDRM Enron Email Data Set in EML, PST, and NSF formats. The corpus is distributed under the Creative Commons Attribution 3.0 United States license.

Attribute the source corpus to ZL Technologies, Inc. (https://www.zlti.com). This release is a modified version because messages and attachments were removed and PST free space was overwritten.

• —ZL Technologies and EDRM announced the v2 corpus in 2010.
• —EDRM announced a privacy-cleansed republication in 2013.

No signed provenance manifest establishes which historical republication supplied the local source mirror. This release therefore documents its own modifications without assuming that the input was already privacy-cleansed.

See LICENSE.md for attribution and modification terms and SHA256SUMS for per-file checksums. Detailed review evidence is kept private because it may contain sensitive material.

Credits

Built with love, by Intellekt.