Yugrathee28/Hinglish-dataset

🇮🇳 Hinglish Dataset — 1.4 Million Samples

Industrial-Grade Code-Mixed NLP Dataset | By ScaleIndia AI · Founder: Yug Rathee

-orange)

This repository contains a 5,000-row teaser sample from the full 1.46 Million+ Hinglish comment dataset built by Scaling YUG (Founder: Yug Rathee(yugrathee28@gmail.com)). Provided strictly for research and evaluation purposes only. Commercial use, redistribution, or production-model training requires explicit written consent from ScaleIndia AI.

📌 What is Hinglish?

Hinglish is a naturally spoken blend of Hindi and English — the dominant code-mixed language used by 500 Million+ people across India's internet. It appears in YouTube comments, WhatsApp chats, Twitter/X posts, and product reviews.

It is one of the most underrepresented yet commercially valuable languages for AI training today. Most LLMs and NLP models perform poorly on Hinglish because no large, clean, labeled dataset existed — until now.

🔷 Full Dataset Overview

⚙️ Full Pipeline Cleaning Report — v7.0-turbo

The full 1.46M dataset was processed through an industrial-grade cleaning pipeline. Below is the verified pipeline output report.

======================================================================
  HINGLISH TURBO CLEANING REPORT  —  v7.0-turbo
  Config hash : 37f130be5a81f15b
  Fast JSON   : orjson
======================================================================
  Runtime     : 43.0 min  (2,582 s)
  Throughput  : 583 rows/s
  Dedup engine: MinHash LSH (full dataset)
======================================================================

📦 Row Count Summary

97.4% retention rate — the input data was already high quality. Only genuinely noisy rows were removed.

🗑️ Removal Breakdown

🏷️ Full Dataset Label Quality

📊 Quality Score Histogram (Kept Rows)

[0.0–0.1]                    0
[0.1–0.2]                    0
[0.2–0.3]                    0
[0.3–0.4]                    0
[0.4–0.5]                    3
[0.5–0.6]                   96
[0.6–0.7]                2,854
[0.7–0.8] █             63,377
[0.8–0.9] ███           225,940
[0.9–1.0] ████████████ 1,174,656

80%+ of kept rows score 0.9–1.0 quality — the dataset is overwhelmingly high-grade text.

📋 5,000-Row Teaser — Audit Report

This teaser file (hinglish_teaser_5k.json) is a stratified, confidence-biased sample of the full dataset.

📝 Text Quality

🏷️ Label Reliability

🎯 Intent Distribution (5k Teaser)

😊 Emotion Distribution (5k Teaser)

✅ Teaser Health Checks

📐 Data Schema

Every row follows this exact structure:

{
    "id": 1,
    "text": "Yaar ye video bahut zyada helpful thi, seriously thank you so much!",
    "intent": "Appreciation",
    "emotion": "Happy",
    "toxicity": "Low",
    "sarcasm": "No",
    "language": "hinglish",
    "quality_score": 0.82,
    "label_confidence": 0.9,
    "label_method": "strong_signal",
    "is_short": false
}

🔬 How This Dataset Was Built

1. 1.Scraping — YouTube comments collected via industrial async scraper at scale
2. 2.Normalization — Unicode normalization, emoji handling, encoding fixes
3. 3.PII Scrubbing — Phone numbers, emails, handle names removed
4. 4.Noise Filtering — Emoji-only, word repetition spam, garbage chars removed
5. 5.Language Filtering — Pure Hindi, pure English, broken translations excluded
6. 6.Quality Scoring — Heuristic scoring on lexical richness, structure, length
7. 7.Deduplication — Exact SHA-256 + near-duplicate MinHash LSH (128 permutations)
8. 8.Labeling — Mega-Regex heuristic labeler across 9 intent + 10 emotion classes
9. 9.Confidence Scoring — Every row assigned label_confidence (0.0–1.0)

💼 Ideal Use Cases

• —🤖 Conversational AI & Chatbot fine-tuning — intent + emotion labels ready to use
• —🧠 Sentiment & emotion analysis in code-mixed Hinglish
• —📚 Low-resource NLP research — one of the largest labeled Hinglish datasets publicly available
• —🔍 Sarcasm & toxicity classification benchmarks
• —🏗️ Pre-training or fine-tuning multilingual / code-mixed LLMs
• —📊 Academic research on South Asian internet language

⚖️ License & Legal

Copyright © 2026 ScaleIndia AI — Yug Rathee. All Rights Reserved.

This dataset is provided STRICTLY for research and evaluation purposes only.

PERMITTED:
  ✅ Viewing and evaluating dataset quality
  ✅ Academic and non-commercial research with attribution
  ✅ Sharing with proper credit to ScaleIndia AI (Yug Rathee)

STRICTLY PROHIBITED WITHOUT EXPLICIT WRITTEN CONSENT:
  ❌ Commercial use of any kind
  ❌ Redistribution, re-uploading, or mirroring this data
  ❌ Training production-grade or commercial AI/ML models
  ❌ Selling, licensing, or sublicensing to third parties
  ❌ Claiming ownership or authorship of this dataset

Violation of these terms may result in legal action under applicable
copyright and intellectual property law.

💰 Purchase the Full Dataset

The full 1,466,926-row Hinglish dataset is available for commercial licensing.

What you get with the full dataset:

• —✅ 1,466,926 cleaned, labeled, deduplicated rows
• —✅ 9 intent classes + 10 emotion classes on every row
• —✅ Toxicity & sarcasm flags
• —✅ Quality score + label confidence score
• —✅ JSON or JSONL format (your choice)
• —✅ Commercial training rights (terms apply)
• —✅ Support for dataset integration queries

📬 Contact ScaleIndia AI

For enterprise licensing, custom dataset slices, or bulk purchases — email with your use case and company name for a quote.

Built with ❤️ by Yug Rathee — ScaleIndia AI — April 2026