Datasetpublic

Noisy77/code-switching-codesaviours-si26-bilal

Roman Urdu-English Code Switching Dataset Dataset Description A manually labeled dataset of 160+ code-switching sentences where Roman Urdu and English are naturally mixed — reflecting how 230 million Pakistanis actually communicate online. Each word in every sentence is tagged with a language label, making this dataset suitable for token-level language identification and code-switching NLP research. Label Meanings Label Description Examples… See the full description on the dataset page: https://huggingface.co/datasets/Noisy77/code-switching-codesaviours-si26-bilal.

sourceHugging Facemitupdated 1mo agoView on Hugging Face

0likes20downloads

Dataset Card

Roman Urdu-English Code Switching Dataset

Dataset Description

A manually labeled dataset of 160+ code-switching sentences where Roman Urdu and English are naturally mixed — reflecting how 230 million Pakistanis actually communicate online.

Each word in every sentence is tagged with a language label, making this dataset suitable for token-level language identification and code-switching NLP research.

Label Meanings

Label	Description	Examples
`URD`	Roman Urdu (Urdu written in Latin script)	Aaj, bohot, hai, mujhe, karo
`ENG`	English	meeting, please, seriously, office
`MIX`	Hybrid or brand names used across both languages	Netflix, Instagram, ChatGPT, Foodpanda

Dataset Format

The dataset is in CSV format with three columns:

Column	Type	Description
`sentence`	string	The full code-switched sentence
`word`	string	Individual word from the sentence
`label`	string	Language label (URD, ENG, or MIX)

Example Entries

sentence	word	label
Aaj ka din bohot busy tha had 3 meetings back to back	Aaj	URD
Aaj ka din bohot busy tha had 3 meetings back to back	ka	URD
Aaj ka din bohot busy tha had 3 meetings back to back	busy	ENG
Aaj ka din bohot busy tha had 3 meetings back to back	meetings	ENG

Collection Methodology

Sentences were crafted to mirror real-world code-switching patterns observed across:

—🐦 Pakistani Twitter/X — searched for common Urdu markers like "kya", "hai", "nahi" mixed with English
—🗨️ Reddit (r/pakistan) — comment sections where bilingual users interact
—📺 YouTube comments — on Pakistani content creators' videos
—💬 WhatsApp-style messaging — casual conversational patterns (anonymized)

Categories Covered

The dataset spans 8 thematic categories to ensure diversity:

1.Daily life & casual conversation (~30 sentences)
2.Work, office & professional (~25 sentences)
3.Food, restaurants & cooking (~20 sentences)
4.Technology & social media (~20 sentences)
5.Education, studies & exams (~20 sentences)
6.Sports, cricket & entertainment (~15 sentences)
7.Shopping, fashion & clothes (~15 sentences)
8.Travel, weather & transport (~15 sentences)

Intended Uses

—Token-level language identification — Train classifiers to detect language switches at the word level
—Code-switching research — Study patterns of how Urdu-English bilingual speakers mix languages
—NER for mixed-language text — Foundation for named entity recognition in code-switched text
—Sentiment analysis — Pre-processing step for sentiment models handling mixed-language input
—Language model fine-tuning — Adapt models to understand and generate code-switched text

How to Load

python

import pandas as pd

# From local file
df = pd.read_csv('dataset.csv')

# Or from HuggingFace
from datasets import load_dataset
dataset = load_dataset('Noisy77/code-switching-codesaviours-si26-bilal')

Limitations

—The dataset focuses on Roman Urdu + English only (not Urdu script)
—Sentences are modeled after real patterns but are not direct copies from social media
—The MIX label is used sparingly and primarily for brand/platform names
—Dataset size is relatively small (160+ sentences) — best used for prototyping or as a seed dataset

Citation

If you use this dataset, please cite:

@dataset{bilal_code_switching_2026,
  title={Roman Urdu-English Code Switching Dataset},
  author={Bilal},
  year={2026},
  publisher={HuggingFace},
  url={https://huggingface.co/datasets/Noisy77/code-switching-codesaviours-si26-bilal}
}

Credit

Built by Bilal during the Code Saviours ML/AI Internship — Batch SI-26.