KhalidAlharbi377/pii-detection-multisource-en-saudi-arabic

PII Detection Multisource EN + Saudi/Arabic

284,619 English examples. 2,088,335 labelled spans. 31 entity types. One label space.

Four public PII datasets, merged into a single schema, plus Saudi and Arabic coverage that none of them had, plus material for two failure modes that matter when you run redaction in production.

Built for OnKith, a privacy first voice assistant that transcribes speech and strips personal information on the device itself, before anything is allowed to leave it.

from datasets import load_dataset

ds = load_dataset("KhalidAlharbi377/pii-detection-multisource-en-saudi-arabic")

Why this exists

OnKith runs speech recognition and PII removal locally on a Raspberry Pi. The idea is that most of a sentence is harmless and a small part of it is not:

"Remind me to call Dr. Sarah Chen at 555-0147 about my appointment on Tuesday."

The intent, setting a reminder for Tuesday, carries almost no risk. The name and the phone number carry nearly all of it. So the pipeline removes the identifiers and passes on the rest:

"Remind me to call [TITLE1] [GIVENNAME1] [SURNAME1] at [TELEPHONENUM1] about my appointment on [DATE_1]."

Doing that on a 4 layer TinyBERT quantized to INT8 means the model has very little capacity to spare, so the training data has to carry the weight. No single public PII dataset was enough. Each one covers different document types, uses its own label names, and none of them contain a single Saudi bank account or Arabic given name. So we combined them and filled the gaps.

What we actually did

We unified the labels. This is the main thing. The four sources between them use somewhere north of a hundred different entity names for the same handful of real concepts. We mapped all of them onto one schema of 31 types, using ai4privacy's naming convention as the target because it was the largest source and its names are the least ambiguous.

We also narrowed the schema on purpose. The sources carry types like biometric identifier, medical record number, API key, CVV, PIN, IPv6 and vehicle identifier. Those are real PII, but they do not appear in spoken living room English, and every extra class costs a model this small real accuracy on the classes that matter. See the limitations section, because this decision has a consequence you need to know about.

We added Saudi and Arabic coverage. 450 rows using Arabic and Muslim given names, Saudi IBANs in correct SAMA format with valid ISO 13616 check digits, 15 digit ZATCA VAT numbers, and real licensed bank names from the Saudi Central Bank list. Every value is fictional. The formats are correct so a model learns the shape, the values point at nobody.

We added spoken password disclosure. 2,145 rows of people saying passwords out loud, across 11 phrasings: telling someone a password, changing one, correcting themselves mid sentence, asking an assistant to remember one. Public PII datasets have passwords sitting in structured documents, not in speech.

We added hard negatives. 146 rows of text that looks like PII and is not, across 32 categories, all with zero spans:

"The rose garden needs water before noon." "ticket 44219 was closed yesterday" "The match ended three one after extra time."

OnKith blocks text when masking looks uncertain, so a false positive is not a cosmetic problem, it is the assistant refusing to work. These rows teach the model to stay quiet.

We augmented the training split. 43,533 rows derived from train rows by swapping entity values and changing case. Every augmented row stays in train next to its parent, so nothing leaks into evaluation.

Where the data comes from

Each source brings something different. ai4privacy brings volume and five English regions. Nemotron brings 30 professional domains, so bills of lading, legal cover pages, medical intake. Gretel brings structured documents, JSON payloads and tokens. Kiji brings conversational text and resumes, which is the closest of the four to how people actually talk.

By generation type: 238,745 rows untouched from the sources, 43,533 augmented, 1,938 generated with AI, 403 with entity values replaced.

Splits

There are two validation sets and that is deliberate. validation is the ai4privacy validation data kept exactly as it was upstream, so numbers stay comparable with anyone else benchmarking on ai4privacy. validation_supplementary covers the other sources, which had no validation split of their own to preserve.

No source_text value appears in more than one split, and every augmented row sits in the same split as the row it came from. There is no contamination between train and test.

The 31 labels

"Total" counts every occurrence, "Rows" counts how many rows contain the label at least once. Median is 6 entities per row, mean 7.34, maximum 100.

Not every source contributes every label. SEX, SALARY and AMOUNT come only from ai4privacy. BUILDINGNUM, TITLE, PASSPORTNUM and DRIVERLICENSENUM come only from ai4privacy and kiji. BANKNAME, CURRENCY and TIMEZONE are mostly the Saudi rows.

Format

Spans are character offsets into source_text, not token tags. Every one of the 2,088,335 spans satisfies source_text[start:end] == value, verified across the whole dataset.

ex = ds["train"][0]
for span in ex["privacy_mask"]:
    print(span["label"], "->", span["value"])
    assert ex["source_text"][span["start"]:span["end"]] == span["value"]

To train a token classifier, align the offsets to your tokenizer with return_offsets_mapping=True.

Loading one split on its own:

test = load_dataset("KhalidAlharbi377/pii-detection-multisource-en-saudi-arabic", split="test")

Limitations

Read this part before you train on it.

Entity types outside the 31 are not annotated, even when they appear in the text. This is the direct consequence of narrowing the schema. A Nemotron row might contain a vehicle identification number sitting unlabelled in the sentence, because vehicle identifier is not one of our types. The same goes for usernames, license plates, API keys, CVV and PIN numbers, IPv6 addresses and medical record numbers. If you need those classes, this dataset will actively teach your model to ignore them.

Annotation coverage inside the 31 types is good but not perfect. We checked by finding email addresses and IPv4 addresses with a regular expression and asking whether a span covered them. About 0.5% of emails and between 2% and 7% of IPv4 addresses are present in the text without a label, depending on the source. It is a small noise floor, but it is not zero.

Saudi and Arabic coverage is a starting point, not a solution. It is 450 rows, 0.2% of the dataset. The text is English throughout. There is no Arabic script anywhere in this dataset. What the Saudi rows give you is exposure to Arabic transliterated names and to Saudi identifier formats inside English sentences. That is genuinely useful and it is not the same thing as an Arabic PII dataset.

The tail labels are too small to trust. SALARY has 2 examples. CURRENCY has 29, AMOUNT 26, TIMEZONE 52, BANKNAME 75. You cannot train these and you cannot meaningfully evaluate them either. Several are missing entirely from one or more splits.

The two validation sets are not interchangeable. validation is 100% ai4privacy and contains almost no ORGANISATION, URL, IPV4, COUNTRY or PASSWORD. If you evaluate only on it you will not see how the model handles the other sources. Use both, or use test.

Kiji has almost no evaluation data. 6,274 rows in train, 6 in validation, none in test. You cannot measure performance on conversational text from this dataset alone.

`uid` is not unique. 2,412 rows share a uid with another row, nearly all from Nemotron, and 313 of those collide across split boundaries. The texts are different, so this is an id collision rather than leakage, but do not use uid to join or deduplicate. Use row_id.

Two leftover columns. partition is a byte for byte copy of split. builder_final_split exists on only the 450 Saudi rows and disagrees with split on 371 of them, because it came from an earlier version of the build. split is the authoritative one. Both columns are kept so nothing is silently dropped.

Small amounts of noise. 113 texts appear more than once, all inside a single split. Two rows have empty text, val_original_000000937 and val_original_000002788. 191 rows carry no entities, of which 146 are the deliberate hard negatives.

`GENDER` and `SEX` are separate labels with 51,023 and 35,638 occurrences. They come from upstream and we kept the distinction rather than guessing at it.

Privacy

Every piece of PII in this dataset is fictional. The four sources are all synthetic corpora, and the material we added is generated. The Saudi IBANs pass checksum validation and the VAT numbers match ZATCA's format, but they belong to no account and no taxpayer. They are shaped correctly and point at nothing. Do not attempt to validate them against any real registry.

License

Released under CC BY 4.0, which is the most restrictive of the four upstream licenses and therefore governs the whole thing.

If you use this dataset you must also credit the sources it is built from. In particular, ai4privacy requires attribution to Ai4Privacy / Ai Suisse SA with a link to their repository.

• —ai4privacy/pii-masking-openpii-1.5m, CC BY 4.0, copyright Ai Suisse SA
• —nvidia/Nemotron-PII, CC BY 4.0
• —gretelai/gretel-pii-masking-en-v1, Apache 2.0
• —DataikuNLP/kiji-pii-training-data, Apache 2.0

Citation

@misc{alharbi2026piimultisource,
  title  = {PII Detection Multisource EN + Saudi/Arabic},
  author = {Alharbi, Khalid},
  year   = {2026},
  url    = {https://huggingface.co/datasets/KhalidAlharbi377/pii-detection-multisource-en-saudi-arabic},
  note   = {Built for the OnKith on-device privacy pipeline. Derived from ai4privacy/pii-masking-openpii-1.5m, nvidia/Nemotron-PII, gretelai/gretel-pii-masking-en-v1 and DataikuNLP/kiji-pii-training-data}
}