Datasetpublic

d3b4g/maldivian-latin-script-corpus

Maldivian Latin-Script Corpus A growing collection of Latin-script text from Maldivian online communities, containing Romanized Dhivehi, English, and code-mixed writing. Data is collected from multiple sources and tagged by origin. Why this dataset is unique Maldivians commonly write Dhivehi phonetically using Latin script rather than switching to the Thaana keyboard. This produces text like: "varah reethi vaahaka eh" → ވަރަށް ރީތި ވާހަކައެއް (very nice story)… See the full description on the dataset page: https://huggingface.co/datasets/d3b4g/maldivian-latin-script-corpus.

sourceHugging Facecc-by-4.0updated 6mo agoView on Hugging Face

0likes12downloads

Dataset Card

Maldivian Latin-Script Corpus

A growing collection of Latin-script text from Maldivian online communities, containing Romanized Dhivehi, English, and code-mixed writing. Data is collected from multiple sources and tagged by origin.

Why this dataset is unique

Maldivians commonly write Dhivehi phonetically using Latin script rather than switching to the Thaana keyboard. This produces text like:

"varah reethi vaahaka eh" → ވަރަށް ރީތި ވާހަކައެއް (very nice story) "maa salhi, next part avahah up kohdhyba" → (so good, please upload next part quickly)

This Romanized Dhivehi is visually indistinguishable from English to any script-based detector, yet it is a distinct and widely-used informal writing system among younger Maldivians.

Sources

Source	Type	Records	Added
esfiya.com	Fiction story comments	~314,000	2026-03

More sources will be added over time — news site comments, YouTube comments, social media. Each record carries a `source` field for filtering by origin.

Dataset fields

Field	Description
`comment_id`	Unique comment ID from the source platform
`post_id`	ID of the article or story this comment belongs to
`date`	Comment timestamp (UTC)
`author_name`	Username of commenter (`null` if anonymous)
`text`	Comment text in Latin script
`parent_id`	ID of parent comment if reply, else `0`
`char_count`	Character count of text
`thaana_ratio`	Ratio of Thaana characters — `0.0` for all current records
`language`	Script label — see language note below
`token_approx`	Approximate token count
`source`	Origin platform (e.g. `esfiya.com`, `vaguthu.mv`)

Statistics (v1.0 — March 2026)

Metric	Value
Total records	~314,000
Sources	1 (esfiya.com)
Date range	October 2012 — March 2026
Thaana ratio	0.0 (all Latin script)
Threaded replies	~38% of records
Avg comment length	~60 characters

Language note

Records in this dataset are written in Latin script and fall into three categories:

—Romanized Dhivehi — Dhivehi language written phonetically in Latin letters. Very common among younger Maldivians. Examples: "varah reethi", "maa salhi", "next part plx", "haadha lahey update vaaleh"
—English — actual English words. Examples: "really nice story", "keep it up", "waiting for next part"
—Mixed — sentences combining both. Example: "varah reethi story, really enjoyed it"

No automatic classifier can reliably separate Romanized Dhivehi from English without a model trained specifically on this writing system. The language field is set to latin for all records. This dataset is itself the primary resource for building such a classifier.

Use cases

—Romanized Dhivehi detection — train a classifier to identify Romanized Dhivehi vs English in Latin-script text
—Code-switching research — study how Maldivians switch between Romanized Dhivehi and English mid-sentence
—Sentiment analysis — reader reactions with naturally implied sentiment (fiction comments carry strong emotional signals)
—Informal language modeling — the most colloquial, everyday Maldivian writing available in any dataset
—Social NLP — threaded reply structure enables conversation and dialogue modeling

Data collection and cleaning

v1.0 — esfiya.com (March 2026)

—Filtered to Latin-script only (thaana_ratio = 0.0)
—Removed noise comments (fewer than 3 unique characters)
—Removed exact duplicates (same text + same post)
—Anonymous comments retained with author_name = null

Changelog

Version	Date	Description
1.0	2026-03	Initial release — esfiya.com comments

Citation

bibtex

@dataset{maldivian_latin_script_corpus_2026,
  title     = {Maldivian Latin-Script Corpus},
  author    = {d3b4g},
  year      = {2026},
  url       = {https://huggingface.co/datasets/d3b4g/maldivian-latin-script-corpus},
  note      = {A growing collection of Latin-script text from Maldivian online communities}
}

License

CC-BY-4.0. Content is user-submitted from the respective source platforms.