jngb-labs/sms-spam

SMS Spam

The SMS Spam Collection, deduplicated.

The original corpus, assembled by Almeida and Gómez Hidalgo in 2011, contains 5,574 SMS messages tagged as ham (legitimate) or spam. About seven percent of the rows are exact duplicates: the same message text appearing more than once. This release removes them and repairs a small set of encoding artifacts. The messages and labels are otherwise unchanged.

5,159 messages remain: 4,517 ham, 642 spam.

What is in it

A single train split. Build your own validation and test splits.

from datasets import load_dataset

ds = load_dataset("jngb-labs/sms-spam", split="train")
print(ds[0])
# {'label': 'ham', 'text': 'Go until jurong point, crazy.. ...'}

What changed from the source

Three things, in order of how much they affect the corpus.

Deduplication. 415 of 5,574 rows removed: 310 ham, 105 spam. The dedup key normalises whitespace, NFKC-folds the text, and lowercases for comparison; the messages themselves are stored in their original casing and punctuation. Spam has a higher duplication rate than ham (about 14 percent vs about 6 percent), consistent with spam being templated and sent at scale. No two messages with different labels collided on the dedup key. The first occurrence of each canonical form wins.

Whitespace cleanup. 424 messages had stray leading or trailing whitespace, or multiple-space runs in the middle. These were stripped or collapsed. Casing untouched.

Encoding repair. A handful of Windows-1252 control bytes survived from an earlier round-trip through a CP1252 environment and rendered as control characters when read as UTF-8. They are mapped to their typographic equivalents (curly quotes, dashes, ellipsis). No actual words changed.

All three are reproducible. scripts/clean.py in this repo, run against the original raw file, produces this dataset byte-for-byte.

What was not changed

Words, punctuation, casing. Label assignments. Anything beyond the three transformations above. There is no train/test split applied here; downstream users construct their own.

What it is good for

Training and benchmarking binary text classifiers, especially the kind of experiments that highlight the gap (or absence of a gap) between simple methods and pretrained transformers. The corpus is small enough that the experiment loops quickly and large enough that the metrics are statistically meaningful. Two derivative models trained on this data are at `jngb-labs/sms-spam-classical` and `jngb-labs/sms-spam-distilbert`, and an interactive comparison demo is at `jngb-labs/sms-spam-classifier`.

What it is not good for

Production deployment against modern spam. The data is from 2011 and earlier. The spam patterns are mostly UK and Singaporean, mostly pre-smartphone. Contemporary smishing campaigns (brand impersonation, URL shorteners, two-factor scams) are underrepresented. Anything trained on this corpus alone will miss patterns invented in the last decade.

Anything that is not English. Anything that requires sender identity, delivery timestamps, or any signal beyond the message body and the label: none of that is in the corpus.

Where it came from

Four sources, blended:

• —About 425 spam messages manually scraped from Grumbletext, a UK forum on which mobile-phone users discussed SMS spam they had received.
• —About 450 ham messages from Caroline Tagg's PhD thesis.
• —About 3,375 ham messages from the NUS SMS Corpus, contributed largely by students at the National University of Singapore.
• —About 1,002 ham and 322 spam messages from José María Gómez Hidalgo's earlier SMS Spam Corpus v.0.1 Big.

Assembled by Almeida and Gómez Hidalgo in 2011. Full attribution preserved in the LICENSE file and in the citation below.

Citation

@inproceedings{Almeida2011SMSSpam,
  author    = {Tiago A. Almeida and Jos\'{e} Mar\'{i}a G\'{o}mez Hidalgo and Akebo Yamakami},
  title     = {Contributions to the study of {SMS} spam filtering: new collection and results},
  booktitle = {Proceedings of the 2011 ACM Symposium on Document Engineering},
  year      = {2011},
}

The original authors ask that users notify them by email at tiago@dt.fee.unicamp.br when the corpus is used in research or production.

Licensing

The original SMS Spam Collection v.1 is distributed under a custom license: free use, attribution required, no warranty. The full text is preserved verbatim in the LICENSE file alongside this README.

Copyright for the corpus content remains with Tiago Agostinho de Almeida and José María Gómez Hidalgo. This deduplicated release does not claim any new copyright over the contents. The cleaning script (scripts/clean.py) is released under MIT.

Changelog

• —2026-05-19. Initial release. Deduplication, whitespace cleanup, encoding repair applied to the SMS Spam Collection v.1.