mteb/IndicNLPNewsClassification

1---2annotations_creators:3- expert-annotated4language:5- guj6- kan7- mal8- mar9- ori10- pan11- tam12- tel13license: cc-by-nc-4.014multilinguality: multilingual15task_categories:16- text-classification17task_ids:18- topic-classification19dataset_info:20- config_name: gu21  features:22  - name: text23    dtype: string24  - name: label25    dtype: int6426  splits:27  - name: train28    num_bytes: 7324946229    num_examples: 1919730  - name: test31    num_bytes: 786319032    num_examples: 204833  download_size: 2999827234  dataset_size: 8111265235- config_name: kn36  features:37  - name: text38    dtype: string39  - name: label40    dtype: int6441  splits:42  - name: train43    num_bytes: 1061695544    num_examples: 204845  - name: test46    num_bytes: 1080116947    num_examples: 204848  download_size: 756679749  dataset_size: 2141812450- config_name: mal51  features:52  - name: text53    dtype: string54  - name: label55    dtype: int6456  splits:57  - name: train58    num_bytes: 675519659    num_examples: 204860  - name: test61    num_bytes: 682530662    num_examples: 204863  download_size: 498359664  dataset_size: 1358050265- config_name: mr66  features:67  - name: text68    dtype: string69  - name: label70    dtype: int6471  splits:72  - name: train73    num_bytes: 1021928474    num_examples: 204875  - name: test76    num_bytes: 1042276877    num_examples: 204878  download_size: 774414779  dataset_size: 2064205280- config_name: ori81  features:82  - name: text83    dtype: string84  - name: label85    dtype: int6486  splits:87  - name: train88    num_bytes: 648387589    num_examples: 204890  - name: test91    num_bytes: 650871692    num_examples: 204893  download_size: 479821094  dataset_size: 1299259195- config_name: pa96  features:97  - name: text98    dtype: string99  - name: label100    dtype: int64101  splits:102  - name: train103    num_bytes: 8074535104    num_examples: 2048105  - name: test106    num_bytes: 2485208107    num_examples: 624108  download_size: 3862134109  dataset_size: 10559743110- config_name: ta111  features:112  - name: text113    dtype: string114  - name: label115    dtype: int64116  splits:117  - name: train118    num_bytes: 9059789119    num_examples: 2048120  - name: test121    num_bytes: 9068394122    num_examples: 2048123  download_size: 6114533124  dataset_size: 18128183125- config_name: tel126  features:127  - name: text128    dtype: string129  - name: label130    dtype: int64131  splits:132  - name: train133    num_bytes: 7704473134    num_examples: 2048135  - name: test136    num_bytes: 7919236137    num_examples: 2048138  download_size: 5799126139  dataset_size: 15623709140configs:141- config_name: gu142  data_files:143  - split: train144    path: gu/train-*145  - split: test146    path: gu/test-*147- config_name: kn148  data_files:149  - split: train150    path: kn/train-*151  - split: test152    path: kn/test-*153- config_name: mal154  data_files:155  - split: train156    path: mal/train-*157  - split: test158    path: mal/test-*159- config_name: mr160  data_files:161  - split: train162    path: mr/train-*163  - split: test164    path: mr/test-*165- config_name: ori166  data_files:167  - split: train168    path: ori/train-*169  - split: test170    path: ori/test-*171- config_name: pa172  data_files:173  - split: train174    path: pa/train-*175  - split: test176    path: pa/test-*177- config_name: ta178  data_files:179  - split: train180    path: ta/train-*181  - split: test182    path: ta/test-*183- config_name: tel184  data_files:185  - split: train186    path: tel/train-*187  - split: test188    path: tel/test-*189tags:190- mteb191- text192---193<!-- adapted from https://github.com/huggingface/huggingface_hub/blob/v0.30.2/src/huggingface_hub/templates/datasetcard_template.md -->194 195<div align="center" style="padding: 40px 20px; background-color: white; border-radius: 12px; box-shadow: 0 2px 10px rgba(0, 0, 0, 0.05); max-width: 600px; margin: 0 auto;">196  <h1 style="font-size: 3.5rem; color: #1a1a1a; margin: 0 0 20px 0; letter-spacing: 2px; font-weight: 700;">IndicNLPNewsClassification</h1>197  <div style="font-size: 1.5rem; color: #4a4a4a; margin-bottom: 5px; font-weight: 300;">An <a href="https://github.com/embeddings-benchmark/mteb" style="color: #2c5282; font-weight: 600; text-decoration: none;" onmouseover="this.style.textDecoration='underline'" onmouseout="this.style.textDecoration='none'">MTEB</a> dataset</div>198  <div style="font-size: 0.9rem; color: #2c5282; margin-top: 10px;">Massive Text Embedding Benchmark</div>199</div>200 201A News classification dataset in multiple Indian regional languages.202 203|               |                                             |204|---------------|---------------------------------------------|205| Task category | t2c                              |206| Domains       | News, Written                               |207| Reference     | https://github.com/AI4Bharat/indicnlp_corpus#indicnlp-news-article-classification-dataset |208 209 210## How to evaluate on this task211 212You can evaluate an embedding model on this dataset using the following code:213 214```python215import mteb216 217task = mteb.get_tasks(["IndicNLPNewsClassification"])218evaluator = mteb.MTEB(task)219 220model = mteb.get_model(YOUR_MODEL)221evaluator.run(model)222```223 224<!-- Datasets want link to arxiv in readme to autolink dataset with paper -->225To learn more about how to run models on `mteb` task check out the [GitHub repitory](https://github.com/embeddings-benchmark/mteb). 226 227## Citation228 229If you use this dataset, please cite the dataset as well as [mteb](https://github.com/embeddings-benchmark/mteb), as this dataset likely includes additional processing as a part of the [MMTEB Contribution](https://github.com/embeddings-benchmark/mteb/tree/main/docs/mmteb).230 231```bibtex232 233@article{kunchukuttan2020indicnlpcorpus,234  author = {Anoop Kunchukuttan and Divyanshu Kakwani and Satish Golla and Gokul N.C. and Avik Bhattacharyya and Mitesh M. Khapra and Pratyush Kumar},235  journal = {arXiv preprint arXiv:2005.00085},236  title = {AI4Bharat-IndicNLP Corpus: Monolingual Corpora and Word Embeddings for Indic Languages},237  year = {2020},238}239 240 241@article{enevoldsen2025mmtebmassivemultilingualtext,242  title={MMTEB: Massive Multilingual Text Embedding Benchmark},243  author={Kenneth Enevoldsen and Isaac Chung and Imene Kerboua and Márton Kardos and Ashwin Mathur and David Stap and Jay Gala and Wissam Siblini and Dominik Krzemiński and Genta Indra Winata and Saba Sturua and Saiteja Utpala and Mathieu Ciancone and Marion Schaeffer and Gabriel Sequeira and Diganta Misra and Shreeya Dhakal and Jonathan Rystrøm and Roman Solomatin and Ömer Çağatan and Akash Kundu and Martin Bernstorff and Shitao Xiao and Akshita Sukhlecha and Bhavish Pahwa and Rafał Poświata and Kranthi Kiran GV and Shawon Ashraf and Daniel Auras and Björn Plüster and Jan Philipp Harries and Loïc Magne and Isabelle Mohr and Mariya Hendriksen and Dawei Zhu and Hippolyte Gisserot-Boukhlef and Tom Aarsen and Jan Kostkan and Konrad Wojtasik and Taemin Lee and Marek Šuppa and Crystina Zhang and Roberta Rocca and Mohammed Hamdy and Andrianos Michail and John Yang and Manuel Faysse and Aleksei Vatolin and Nandan Thakur and Manan Dey and Dipam Vasani and Pranjal Chitale and Simone Tedeschi and Nguyen Tai and Artem Snegirev and Michael Günther and Mengzhou Xia and Weijia Shi and Xing Han Lù and Jordan Clive and Gayatri Krishnakumar and Anna Maksimova and Silvan Wehrli and Maria Tikhonova and Henil Panchal and Aleksandr Abramov and Malte Ostendorff and Zheng Liu and Simon Clematide and Lester James Miranda and Alena Fenogenova and Guangyu Song and Ruqiya Bin Safi and Wen-Ding Li and Alessia Borghini and Federico Cassano and Hongjin Su and Jimmy Lin and Howard Yen and Lasse Hansen and Sara Hooker and Chenghao Xiao and Vaibhav Adlakha and Orion Weller and Siva Reddy and Niklas Muennighoff},244  publisher = {arXiv},245  journal={arXiv preprint arXiv:2502.13595},246  year={2025},247  url={https://arxiv.org/abs/2502.13595},248  doi = {10.48550/arXiv.2502.13595},249}250 251@article{muennighoff2022mteb,252  author = {Muennighoff, Niklas and Tazi, Nouamane and Magne, Lo{\"\i}c and Reimers, Nils},253  title = {MTEB: Massive Text Embedding Benchmark},254  publisher = {arXiv},255  journal={arXiv preprint arXiv:2210.07316},256  year = {2022}257  url = {https://arxiv.org/abs/2210.07316},258  doi = {10.48550/ARXIV.2210.07316},259}260```261 262# Dataset Statistics263<details>264  <summary> Dataset Statistics</summary>265 266The following code contains the descriptive statistics from the task. These can also be obtained using:267 268```python269import mteb270 271task = mteb.get_task("IndicNLPNewsClassification")272 273desc_stats = task.metadata.descriptive_stats274```275 276```json277{278    "test": {279        "num_samples": 14960,280        "number_of_characters": 23509208,281        "number_texts_intersect_with_train": 298,282        "min_text_length": 0,283        "average_text_length": 1571.4711229946524,284        "max_text_length": 25461,285        "unique_text": 13944,286        "unique_labels": 4,287        "labels": {288            "1": {289                "count": 4723290            },291            "0": {292                "count": 4509293            },294            "2": {295                "count": 4490296            },297            "3": {298                "count": 1238299            }300        },301        "hf_subset_descriptive_stats": {302            "gu": {303                "num_samples": 2048,304                "number_of_characters": 3016704,305                "number_texts_intersect_with_train": 171,306                "min_text_length": 307,307                "average_text_length": 1473.0,308                "max_text_length": 11982,309                "unique_text": 1983,310                "unique_labels": 3,311                "labels": {312                    "1": {313                        "count": 687314                    },315                    "0": {316                        "count": 705317                    },318                    "2": {319                        "count": 656320                    }321                }322            },323            "kn": {324                "num_samples": 2048,325                "number_of_characters": 4129549,326                "number_texts_intersect_with_train": 7,327                "min_text_length": 303,328                "average_text_length": 2016.38134765625,329                "max_text_length": 24145,330                "unique_text": 2027,331                "unique_labels": 3,332                "labels": {333                    "0": {334                        "count": 657335                    },336                    "1": {337                        "count": 677338                    },339                    "2": {340                        "count": 714341                    }342                }343            },344            "mal": {345                "num_samples": 2048,346                "number_of_characters": 2518131,347                "number_texts_intersect_with_train": 11,348                "min_text_length": 188,349                "average_text_length": 1229.55615234375,350                "max_text_length": 13458,351                "unique_text": 2038,352                "unique_labels": 4,353                "labels": {354                    "0": {355                        "count": 496356                    },357                    "3": {358                        "count": 515359                    },360                    "2": {361                        "count": 501362                    },363                    "1": {364                        "count": 536365                    }366                }367            },368            "mr": {369                "num_samples": 2048,370                "number_of_characters": 3951866,371                "number_texts_intersect_with_train": 11,372                "min_text_length": 428,373                "average_text_length": 1929.6220703125,374                "max_text_length": 25461,375                "unique_text": 2043,376                "unique_labels": 3,377                "labels": {378                    "2": {379                        "count": 666380                    },381                    "1": {382                        "count": 701383                    },384                    "0": {385                        "count": 681386                    }387                }388            },389            "tel": {390                "num_samples": 2048,391                "number_of_characters": 3037579,392                "number_texts_intersect_with_train": 30,393                "min_text_length": 301,394                "average_text_length": 1483.19287109375,395                "max_text_length": 11974,396                "unique_text": 1991,397                "unique_labels": 3,398                "labels": {399                    "0": {400                        "count": 678401                    },402                    "1": {403                        "count": 715404                    },405                    "2": {406                        "count": 655407                    }408                }409            },410            "ori": {411                "num_samples": 2048,412                "number_of_characters": 2471544,413                "number_texts_intersect_with_train": 0,414                "min_text_length": 323,415                "average_text_length": 1206.80859375,416                "max_text_length": 8448,417                "unique_text": 2048,418                "unique_labels": 4,419                "labels": {420                    "0": {421                        "count": 490422                    },423                    "3": {424                        "count": 560425                    },426                    "1": {427                        "count": 523428                    },429                    "2": {430                        "count": 475431                    }432                }433            },434            "pa": {435                "num_samples": 624,436                "number_of_characters": 998432,437                "number_texts_intersect_with_train": 0,438                "min_text_length": 0,439                "average_text_length": 1600.051282051282,440                "max_text_length": 8727,441                "unique_text": 623,442                "unique_labels": 4,443                "labels": {444                    "2": {445                        "count": 144446                    },447                    "1": {448                        "count": 169449                    },450                    "0": {451                        "count": 148452                    },453                    "3": {454                        "count": 163455                    }456                }457            },458            "ta": {459                "num_samples": 2048,460                "number_of_characters": 3385403,461                "number_texts_intersect_with_train": 3,462                "min_text_length": 595,463                "average_text_length": 1653.02880859375,464                "max_text_length": 8002,465                "unique_text": 2046,466                "unique_labels": 3,467                "labels": {468                    "1": {469                        "count": 715470                    },471                    "0": {472                        "count": 654473                    },474                    "2": {475                        "count": 679476                    }477                }478            }479        }480    },481    "train": {482        "num_samples": 33533,483        "number_of_characters": 50589676,484        "number_texts_intersect_with_train": null,485        "min_text_length": 245,486        "average_text_length": 1508.6534458593028,487        "max_text_length": 19536,488        "unique_text": 29951,489        "unique_labels": 4,490        "labels": {491            "0": {492                "count": 10600493            },494            "1": {495                "count": 10583496            },497            "2": {498                "count": 10842499            },500            "3": {501                "count": 1508502            }503        },504        "hf_subset_descriptive_stats": {505            "gu": {506                "num_samples": 19197,507                "number_of_characters": 28120011,508                "number_texts_intersect_with_train": null,509                "min_text_length": 299,510                "average_text_length": 1464.8127832473824,511                "max_text_length": 12354,512                "unique_text": 17695,513                "unique_labels": 3,514                "labels": {515                    "0": {516                        "count": 6345517                    },518                    "1": {519                        "count": 6390520                    },521                    "2": {522                        "count": 6462523                    }524                }525            },526            "kn": {527                "num_samples": 2048,528                "number_of_characters": 4059197,529                "number_texts_intersect_with_train": null,530                "min_text_length": 297,531                "average_text_length": 1982.02978515625,532                "max_text_length": 16016,533                "unique_text": 2029,534                "unique_labels": 3,535                "labels": {536                    "2": {537                        "count": 709538                    },539                    "1": {540                        "count": 682541                    },542                    "0": {543                        "count": 657544                    }545                }546            },547            "mal": {548                "num_samples": 2048,549                "number_of_characters": 2490600,550                "number_texts_intersect_with_train": null,551                "min_text_length": 336,552                "average_text_length": 1216.11328125,553                "max_text_length": 9018,554                "unique_text": 2039,555                "unique_labels": 4,556                "labels": {557                    "0": {558                        "count": 519559                    },560                    "3": {561                        "count": 531562                    },563                    "2": {564                        "count": 531565                    },566                    "1": {567                        "count": 467568                    }569                }570            },571            "mr": {572                "num_samples": 2048,573                "number_of_characters": 3872775,574                "number_texts_intersect_with_train": null,575                "min_text_length": 479,576                "average_text_length": 1891.00341796875,577                "max_text_length": 19536,578                "unique_text": 2046,579                "unique_labels": 3,580                "labels": {581                    "1": {582                        "count": 645583                    },584                    "2": {585                        "count": 701586                    },587                    "0": {588                        "count": 702589                    }590                }591            },592            "tel": {593                "num_samples": 2048,594                "number_of_characters": 2953411,595                "number_texts_intersect_with_train": null,596                "min_text_length": 309,597                "average_text_length": 1442.09521484375,598                "max_text_length": 12083,599                "unique_text": 1983,600                "unique_labels": 3,601                "labels": {602                    "0": {603                        "count": 679604                    },605                    "1": {606                        "count": 706607                    },608                    "2": {609                        "count": 663610                    }611                }612            },613            "ori": {614                "num_samples": 2048,615                "number_of_characters": 2467807,616                "number_texts_intersect_with_train": null,617                "min_text_length": 303,618                "average_text_length": 1204.98388671875,619                "max_text_length": 8417,620                "unique_text": 2048,621                "unique_labels": 4,622                "labels": {623                    "2": {624                        "count": 549625                    },626                    "1": {627                        "count": 500628                    },629                    "3": {630                        "count": 476631                    },632                    "0": {633                        "count": 523634                    }635                }636            },637            "pa": {638                "num_samples": 2048,639                "number_of_characters": 3248339,640                "number_texts_intersect_with_train": null,641                "min_text_length": 245,642                "average_text_length": 1586.10302734375,643                "max_text_length": 17538,644                "unique_text": 2047,645                "unique_labels": 4,646                "labels": {647                    "1": {648                        "count": 515649                    },650                    "3": {651                        "count": 501652                    },653                    "0": {654                        "count": 515655                    },656                    "2": {657                        "count": 517658                    }659                }660            },661            "ta": {662                "num_samples": 2048,663                "number_of_characters": 3377536,664                "number_texts_intersect_with_train": null,665                "min_text_length": 434,666                "average_text_length": 1649.1875,667                "max_text_length": 10438,668                "unique_text": 2047,669                "unique_labels": 3,670                "labels": {671                    "2": {672                        "count": 710673                    },674                    "1": {675                        "count": 678676                    },677                    "0": {678                        "count": 660679                    }680                }681            }682        }683    }684}685```686 687</details>688 689---690*This dataset card was automatically generated using [MTEB](https://github.com/embeddings-benchmark/mteb)*