fatymahaly/urdu_rag_dataset.csv

Dataset Card for Urdu RAG Knowledge Base

Dataset Overview

This dataset is designed specifically to bootstrap and evaluate Retrieval-Augmented Generation (RAG) applications, search systems, and semantic retrieval pipelines using the Urdu language. It contains 185 clean, structured, and informative text chunks covering a wide array of domains.

• —Language: Urdu (ur)
• —Script: Nastaliq / Arabic script (Unicode UTF-8)
• —Total Rows: 185 chunks
• —Format: CSV (id, title, content)
• —Primary Use Case: Vector embedding ingestion, semantic search, retrieval testing, and knowledge-base seeding for Urdu LLM applications.

Dataset Structure

The dataset follows a tabular CSV schema with three primary columns:

Content Domains Covered

The 185 entries span a balanced mix of topics to ensure diverse semantic representation in vector databases:

1. 1.Geography & Provinces: Detailed entries on Punjab, Sindh, Khyber Pakhtunkhwa, Balochistan, Gilgit-Baltistan, Azad Kashmir, and major cities (Lahore, Karachi, Peshawar, Quetta, Islamabad, Multan, Faisalabad, etc.).
2. 2.History & National Heritage: Landmark historical events, archaeological sites (Mohenjo-daro, Harappa, Taxila, Rohtas Fort), and the Pakistan Movement.
3. 3.Culture, Art & Literature: Prominent Urdu poets and writers (Mir, Ghalib, Iqbal, Faiz, Manto), traditional attire, regional festivals, music (Sufi/Qawwali), and crafts.
4. 4.Science, Technology & RAG Development: Modern technical articles covering IT outsourcing, AI, Natural Language Processing (NLP), vector embeddings, RAG architectures, and programming.
5. 5.Economy, Agriculture & Industry: Key economic drivers, textile sectors, exports (rice, mangoes, sports goods, surgical instruments), CPEC, and energy projects.
6. 6.Sports & Achievements: National sports, cricket world cups, squash legends (Jahangir Khan, Jansher Khan), and athletics (Arshad Nadeem).

How to Load and Use

You can easily ingest this dataset into your Python-based RAG pipeline (using Pandas, LangChain, or LlamaIndex) with the following snippet:

import pandas as pd

# Load the dataset
df = pd.read_csv("urdu_rag_dataset.csv")

# Preview the first few rows
print(df.head())

# Iterate through chunks to create documents for a vector store
from langchain.schema import Document

documents = [
    Document(page_content=row['content'], metadata={"id": row['id'], "title": row['title']})
    for _, row in df.iterrows()
]

print(f"Loaded {len(documents)} documents for RAG vectorization.")