datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
IndicQA\IndicGenBench_xquad_in
Dataset Card for Dataset Name
This repository contains the XQuAD-IN dataset released as a part of the paper "IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languages"
Paper Link: https://arxiv.org/abs/2404.16816
Dataset Details
Overview
IndicGenBench is a multilingual, multi-way parallel benchmark for measuring language generation capabilities across diverse user-facing tasks in 29 Indic languages spanning 13… See the full description on the dataset page: https://huggingface.co/datasets/google/IndicGenBench_xquad_in.mmlu-indic
Indic MMLU Dataset
A multilingual version of the Massive Multitask Language Understanding (MMLU) benchmark, translated from English into 10 Indian languages.
This version contains the translations of the development and test sets only.
Languages Covered
The dataset includes translations in the following languages:
Bengali (bn)
Gujarati (gu)
Hindi (hi)
Kannada (kn)
Marathi (mr)
Malayalam (ml)
Oriya (or)
Punjabi (pa)
Tamil (ta)
Telugu (te)
Task Format
Each… See the full description on the dataset page: https://huggingface.co/datasets/sarvamai/mmlu-indic.IndicGenBench_xorqa_in
Dataset Card for Dataset Name
This repository contains the XorQA-IN dataset released as a part of the paper "IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languages"
Paper Link: https://arxiv.org/abs/2404.16816
Dataset Details
Overview
IndicGenBench is a multilingual, multi-way parallel benchmark for measuring language generation capabilities across diverse user-facing tasks in 29 Indic languages spanning 13… See the full description on the dataset page: https://huggingface.co/datasets/google/IndicGenBench_xorqa_in.indic-swim-ir-cross-lingual
Dataset Card for Indic SWIM-IR (Cross-lingual)
This is the cross-lingual Indic subset of the SWIM-IR dataset, where the query generated is in the Indo-European language and the passage is in English.
The SWIM-IR dataset is available as CC-BY-SA 4.0. 18 languages (including English) are available in the cross-lingual dataset.
For full details of the dataset, please read our upcoming NAACL 2024 paper and check out our website.
What is SWIM-IR?
SWIM-IR dataset is a… See the full description on the dataset page: https://huggingface.co/datasets/nthakur/indic-swim-ir-cross-lingual.IndicQuest-v2
IndicQuest v2
A gold-standard multilingual question-answering benchmark for evaluating the India-specific factual knowledge of Large Language Models. 3,471 curriculum-grounded English question–answer pairs across nine domains, translated into 19 Indic languages: 69,420 parallel pairs across 20 languages.
More details can be found in our paper.
Dataset structure
One CSV per language, named <language>.csv (english.csv, hindi.csv, marathi.csv, …). Every file has the… See the full description on the dataset page: https://huggingface.co/datasets/l3cube-pune/IndicQuest-v2.Indic-Rag-Suite
🌏 Multilingual Indic RAG Suite
A comprehensive multilingual question-answering dataset covering 18 Indian languages with 21,439,886 total samples, designed for RAG (Retrieval-Augmented Generation) applications and multilingual NLP research.
🚀 Quick Start
from datasets import load_dataset
# Load specific language (recommended)
dataset = load_dataset("ai4bharat/Indic-Rag-Suite", "as")
train_data = dataset['train']
print(f"Loaded {len(train_data)} samples")
# Access… See the full description on the dataset page: https://huggingface.co/datasets/ai4bharat/Indic-Rag-Suite.indic-queries-2026
MAST Indic Queries 2026
This dataset contains the Indic query set for MAST @ FIRE 2026, the Multilingual Agentic Search Track. MAST evaluates whether multilingual agentic search systems can answer complex questions posed in different languages.
MAST builds on BrowseComp-Plus (ACL 2026), a reproducible and verifiable extension of BrowseComp with challenging English queries, a verified English corpus of roughly 100K web-sourced documents, and human judgments. In the 2026 MAST… See the full description on the dataset page: https://huggingface.co/datasets/mast-benchmark/indic-queries-2026.IndicKLAR
IndicKLAR
IndicKLAR (Indic Knowledge and Language-consistency Assessment Resource) is a factual
question-answering dataset for evaluating cross-lingual consistency of LLMs across 18
Indic languages and English, plus romanized/code-mixed variants.
Paper preprint: arXiv:2605.29637 · Code:
IndicKLAR_EMNLP_2026
Summary
78,570 samples — 2,619 samples × 30 language/script variants
20 relations (e.g. capital, place_of_birth, official_language, occupation)
19… See the full description on the dataset page: https://huggingface.co/datasets/debajyotimaz/IndicKLAR.indic-squad
IndicSQuAD Dataset
Dataset Description
IndicSQuAD is a comprehensive multilingual extractive Question Answering (QA) dataset covering nine major Indic languages: Hindi, Bengali, Tamil, Telugu, Marathi, Gujarati, Urdu, Kannada, Oriya, and Malayalam. It's systematically derived from the popular English SQuAD (Stanford Question Answering Dataset).
The rapid progress in QA systems has predominantly benefited high-resource languages, leaving Indic languages significantly… See the full description on the dataset page: https://huggingface.co/datasets/l3cube-pune/indic-squad.IndicMSMARCO
🔍 IndicMSMARCO: Multilingual Information Retrieval Benchmark
A comprehensive multilingual variant of MS MARCO specifically tailored for Indian languages, featuring carefully selected queries and corresponding passages with high-quality translations.
🚀 Quick Start - Load Individual Languages
from datasets import load_dataset
# Load ONLY Hindi data (fast and efficient!)
hindi_data = load_dataset("ai4bharat/IndicMSMARCO", "hi")
print(f"Hindi queries:… See the full description on the dataset page: https://huggingface.co/datasets/ai4bharat/IndicMSMARCO.mmlu-indic
Indic MMLU Dataset
A multilingual version of the Massive Multitask Language Understanding (MMLU) benchmark, translated from English into 10 Indian languages.
This version contains the translations of the development and test sets only.
Languages Covered
The dataset includes translations in the following languages:
Bengali (bn)
Gujarati (gu)
Hindi (hi)
Kannada (kn)
Marathi (mr)
Malayalam (ml)
Oriya (or)
Punjabi (pa)
Tamil (ta)
Telugu (te)
Task Format
Each… See the full description on the dataset page: https://huggingface.co/datasets/mygitphase/mmlu-indic.IndicParam
Dataset Card for IndicParam
Paper | Code
Dataset Summary
IndicParam is a graduate-level benchmark designed to evaluate Large Language Models (LLMs) on their understanding of low- and extremely low-resource Indic languages.The dataset contains 13,207 multiple-choice questions (MCQs) across 11 Indic languages, plus a separate Sanskrit–English code-mixed set, all sourced from official UGC-NET language question papers and answer keys.
Supported Tasks… See the full description on the dataset page: https://huggingface.co/datasets/bharatgenai/IndicParam.IndicDB
IndicDB — Multilingual Text-to-SQL Benchmark for Indian Languages
IndicDB is a comprehensive multilingual Text-to-SQL benchmark for evaluating cross-lingual semantic parsing across diverse Indic language families. Questions are posed in 7 languages while the underlying schemas and values remain in English — simultaneously stressing translation, schema linking, value grounding, and multi-table join reasoning.
Schemas are sourced from real Indian open-data platforms (NDAP and… See the full description on the dataset page: https://huggingface.co/datasets/roshankaranth/IndicDB.IndicVault
Indic Vault — everyday Indian language QA pairs, tuned for chatbots & voice agents.
🧾 Overview
Indic Vault is a high-quality, instruction-tuned dataset featuring question-answer pairs crafted in the contemporary, everyday language spoken across India in 2025. Unlike traditional datasets that lean heavily on formal or outdated linguistic styles, Indic Vault captures the authentic, colloquial expressions used in daily conversations, making it ideal for building AI… See the full description on the dataset page: https://huggingface.co/datasets/maya-research/IndicVault.boolq-indic
Indic BoolQ Dataset
A multilingual version of the BoolQ (Boolean Questions) dataset, translated from English into 10 Indian languages.
It is a question-answering dataset for yes/no questions containing ~12k naturally occurring questions.
Languages Covered
The dataset includes translations in the following languages:
Bengali (bn)
Gujarati (gu)
Hindi (hi)
Kannada (kn)
Marathi (mr)
Malayalam (ml)
Oriya (or)
Punjabi (pa)
Tamil (ta)
Telugu (te)
Dataset Format
Each… See the full description on the dataset page: https://huggingface.co/datasets/sarvamai/boolq-indic.IndicConBench
IndicConBench: A Multilingual Constitutional Reasoning Benchmark for India
IndicConBench is a publication-grade, research-quality multilingual constitutional reasoning benchmark for India. It is specifically designed to evaluate the factual accuracy, reasoning capacity, structural retrieval capability, and legal intelligence of Large Language Models (LLMs) on the Constitution of India in both English and Hindi.
Developed by Vikhram S and published under Vikhram Labs, this… See the full description on the dataset page: https://huggingface.co/datasets/Vikhram-S/IndicConBench.Indic-KCC-Agri-Advisory-Benchmark
Indic-KCC-Agri-Advisory-Benchmark
⚠️ Benchmark only — not agronomic advice. This dataset and its reference
answers exist to score language models, not to be used as real farming
guidance. KCC references are noisy call-centre transcripts (see Status and
caveats); do not act on any answer, reference or candidate, as agricultural
advice.
Open-ended agricultural-advisory question answering in 11 Indian languages,
built from real farmer questions and the advisory answers… See the full description on the dataset page: https://huggingface.co/datasets/sthanika-ai/Indic-KCC-Agri-Advisory-Benchmark.mmlu_indic
Indic MMLU Dataset
A multilingual version of the Massive Multitask Language Understanding (MMLU) benchmark, translated from English into 10 Indian languages.
This version contains the translations of the development and test sets only.
Languages Covered
The dataset includes translations in the following languages:
Bengali (bn)
Gujarati (gu)
Hindi (hi)
Kannada (kn)
Marathi (mr)
Malayalam (ml)
Oriya (or)
Punjabi (pa)
Tamil (ta)
Telugu (te)
Task Format
Each… See the full description on the dataset page: https://huggingface.co/datasets/mygitphase/mmlu_indic.IndicQA\IndicRxNorm-LexMap-15K
IndicRxNorm-LexMap-15K
Dataset Summary
IndicRxNorm-LexMap-15K is a multilingual Indic medicine terminology instruction dataset for medicine-name understanding, RxNorm normalization, RxCUI entity linking, structured drug-field extraction, and safe non-prescriptive clinical terminology tasks.
This Hugging Face repository contains two dataset configurations:
Config
File
Role
multilingual_rxnorm_normalization
multilingual_rxnorm_normalization.jsonl
Primary adapted… See the full description on the dataset page: https://huggingface.co/datasets/AXONVERTEX-AI-RESEARCH/IndicRxNorm-LexMap-15K.indicbench
indicbench
A balanced evaluation suite for LLMs and VLMs across 25 languages — all 22
scheduled Indian languages, plus Bhoti, Sinhala and English. Run it with
github.com/dhruvil237/indicbench.
uv run indicbench download --split main --out data/indicbench
uv run indicbench run --data data/indicbench \
--backend vllm --base-url http://localhost:8000/v1 --model <your-model>
This is a personal, non-commercial research release by an individual, not an
organization's publication.… See the full description on the dataset page: https://huggingface.co/datasets/dhruvil237/indicbench.IndicConBench
IndicConBench: A Multilingual Constitutional Reasoning Benchmark for India
IndicConBench is a publication-grade, research-quality multilingual constitutional reasoning benchmark for India. It is specifically designed to evaluate the factual accuracy, reasoning capacity, structural retrieval capability, and legal intelligence of Large Language Models (LLMs) on the Constitution of India in both English and Hindi.
Developed by Vikhram S and published under Vikhram Labs, this… See the full description on the dataset page: https://huggingface.co/datasets/vikhram-labs/IndicConBench.kisan-advisory-multilingual-indic
Kisan Advisory (Hindi / Punjabi / English)
Real farmer questions and Farm Tele Advisor answers from India's government
Kisan Call Centre helpline, adapted with AutoScientist, expanded into
Hindi and Punjabi, and filtered so that every row provably preserves the
agrochemical doses in its source note.
Rows (after dose filtering)
6,232
Language split
2,064 en / 2,726 hi / 1,442 pa
Quality grade
E → C (3.0 → 5.8)
Relative improvement
+93.3%
Percentile
13.8… See the full description on the dataset page: https://huggingface.co/datasets/tojpaj/kisan-advisory-multilingual-indic.indic_reasoning
Indic Reasoning
The Indic Reasoning Dataset (~500M tokens, 592k examples) is a high-quality, large-scale open-source resource created using advanced distillation techniques. It is designed to train and evaluate reasoning-capable AI systems with a strong emphasis on complex reasoning, structured chain-of-thought (CoT), and culturally relevant content.
This domain-rich corpus integrates Indian cultural, legal, historical, philosophical, and social contexts with global knowledge… See the full description on the dataset page: https://huggingface.co/datasets/169Pi/indic_reasoning.indiccare-triage-rx-v0.1
Version note
This is a v0.1 pilot release created for the Adaptive Data Challenge. The dataset demonstrates the full pipeline and includes accepted/rejected quality gates, but additional manual review and scaling are planned before a larger release.
IndicCare-Triage-Rx
Code Author: Krishnendu DasguptaUsecase: Adaptive Challenge
IndicCare-Triage-Rx is a multilingual Indic-language primary-care safety-triage research dataset. It focuses on red-flag detection, care-urgency… See the full description on the dataset page: https://huggingface.co/datasets/AXONVERTEX-AI-RESEARCH/indiccare-triage-rx-v0.1.IndicIPRExtractiveQA
IndicIPR-QA
A Benchmark for Extractive Question Answering on Indian IPR Laws
IndicIPR-QA is an English-language benchmark for extractive question answering over Indian Intellectual Property Rights (IPR) statutes.
It contains 4,462 question–answer pairs derived from 1,116 statutory contexts covering all seven principal Indian IPR Acts.
Every gold answer is an extractive span grounded directly in its source statutory context, making the benchmark suitable for… See the full description on the dataset page: https://huggingface.co/datasets/niloycste68/IndicIPRExtractiveQA.IndicQuest
L3Cube-IndicQuest: A Benchmark Question Answering Dataset for Evaluating Knowledge of LLMs in Indic Context (LLM Factual Accuracy Benchmark)
L3Cube-IndicQuest is a dataset comprising 4,000 question-answer pairs across 20 languages, including English, Assamese, Bengali, Dogri, Gujarati, Hindi, Kannada, Konkani, Maithili, Malayalam, Marathi, Meitei (Manipuri), Nepali, Odia, Punjabi, Sanskrit, Sindhi, Tamil, Telugu, and Urdu. This dataset is designed to assess the knowledge… See the full description on the dataset page: https://huggingface.co/datasets/l3cube-pune/IndicQuest.Annex_7_Educational_Indicators_2024-2025
Grounded Educational Indicators — Nepali SFT Dataset
File: grounded_education_indicators_nepali_sft.jsonl
Records: 10,091
Language: Nepali (ne / ISO 639-3 npi), Devanagari script
License: Apache-2.0 (permissive)
Format: JSON Lines, ShareGPT-style conversational (human / gpt turns)
Task type: Grounded question answering over structured (tabular) educational statistics
Size on disk: ~15.5 MB
1. Overview
This dataset is a supervised fine-tuning (SFT) corpus of 10… See the full description on the dataset page: https://huggingface.co/datasets/Somtharu181coder/Annex_7_Educational_Indicators_2024-2025.indic-reg-bench
Dataset Card — indic-reg-bench
Status: under construction. The gold set does not exist yet. This card describes what is built, what is not, and the decisions taken so far. It will be wrong in places until labelling is finished; it is published early so the construction method is auditable rather than reconstructed afterwards.
What this is
A benchmark for Indian regulatory document understanding, built on SEBI (Securities and Exchange Board of India) enforcement… See the full description on the dataset page: https://huggingface.co/datasets/siddharthgaur/indic-reg-bench.