Datasetpublic

Podric/prowl-secrets-corpus

Prowl secrets corpus A labeled corpus for training and evaluating secret detectors (credentials, API keys, tokens, database URIs, private keys, and passwords) across code, Jira tickets, Confluence pages, chat, and logs, in multiple languages. It is the training data behind Prowl and its stage-3 encoder. 503,027 records: 181,530 positive, 321,497 negative. No live credentials. Every value is synthetic, format-preserving-obfuscated, or drawn from a public test fixture. The… See the full description on the dataset page: https://huggingface.co/datasets/Podric/prowl-secrets-corpus.

sourceHugging Facecc-by-nc-4.0updated 3mo agoView on Hugging Face

2likes594downloads

Dataset Card

Prowl secrets corpus

A labeled corpus for training and evaluating secret detectors (credentials, API keys, tokens, database URIs, private keys, and passwords) across code, Jira tickets, Confluence pages, chat, and logs, in multiple languages. It is the training data behind Prowl and its stage-3 encoder.

503,027 records: 181,530 positive, 321,497 negative.

No live credentials. Every value is synthetic, format-preserving-obfuscated, or drawn from a public test fixture. The corpus is built so it can be handled without leaking a real secret.

Loading

python

from datasets import load_dataset

ds = load_dataset("Podric/prowl-secrets-corpus")                    # full corpus (default config)
bench = load_dataset("Podric/prowl-secrets-corpus", "prowlbench")   # the held-out evaluation config

ex = ds["train"][0]   # {"text", "value", "label_binary", "label_type", "span", "source", "origin", "context"}

Fields

field	type	meaning
`text`	string	the fragment to classify (code line, config value, ticket/log/chat text)
`value`	string \	null	the secret substring, when positive
`label_binary`	0 \	1	contains a secret
`label_type`	string \	null	secret type (`aws_access_key_id`, `stripe_secret_key`, `generic_password`, ...)
`span`	[int,int] \	null	`[start,end)` of `value` inside `text` (for NER)
`source`	string	channel: `code` \	`jira` \	`confluence` \	`log` \	`slack`
`origin`	string	provenance tag (see below)
`context`	string \	null	surrounding context, when available

Composition by channel

code	Jira	Confluence	log	chat
302,153	97,122	80,092	19,883	3,777

Non-code (tickets, wiki, logs, chat) is ~40% of the corpus, the realistic-leak surface that code-only datasets omit.

Provenance: where each part comes from

origin	share	source	how it was made
`synthetic`	73.8%	original	format- and checksum-correct fake secrets generated per type, injected into templated carriers at known spans
`creddata`	8.4%	Samsung/CredData	real-world credential samples, character-class obfuscated so format survives but no real value does
`hf`	6.8%	ai4privacy/pii-masking-200k, nvidia/Nemotron-PII	synthetic PII prose; passwords → positives, other PII → hard negatives (`neg_pii`)
`fleet*` / `authored` / `carrier`	10.9%	original	synthetic carriers: code in 17 languages, 44 config formats, ticket/wiki/chat/log documents, hard negatives, obfuscation
`fixture`	<0.1%	gitleaks / detect-secrets / whispers test suites (public)	imported public test fixtures

The synthetic, fleet*, authored, and carrier portions are original to this project. The creddata, hf, and fixture portions are derived from the linked upstream datasets and remain under their respective licenses (CredData; ai4privacy and Nemotron-PII per their HF cards; the tool fixtures per each project's license). Attribute the originals if you redistribute.

Splits and leakage control

The corpus is meant to be split value-disjoint (no secret value appears in both train and eval) and origin-disjoint: the real-secret sources (creddata, hf) are held out so a validation score measures generalization, not memorization. The encoder's operating threshold is calibrated on such a held-out split.

`prowlbench` config

The second config is ProwlBench: 3,843 leakage-safe evaluation cases (structured tokens, generic high-entropy keys, multilingual free-form prose) with per-case tier and language, used to compare scanners as real subprocesses. It is value-disjoint from corpus.

Intended use

Training and evaluating secret/credential detectors and DLP classifiers. Not a source of real credentials and not usable as one.

License

Noncommercial use only (CC BY-NC 4.0). Not for use in commercial products. The original portions (synthetic, fleet*, authored, carrier) are released under CC BY-NC 4.0. The derived portions (creddata, hf, fixture) stay under their upstream licenses, listed in the provenance table. Attribute the originals if you redistribute.

Citation

bibtex

@misc{prowl_secrets_corpus,
  title        = {Prowl secrets corpus},
  author       = {Prowl},
  year         = {2026},
  howpublished = {Hugging Face Datasets},
  url          = {https://huggingface.co/datasets/Podric/prowl-secrets-corpus}
}