mteb/multilingual-sentiment-classification

1---2annotations_creators:3- derived4language:5- ara6- bam7- bul8- cmn9- cym10- deu11- dza12- ell13- eng14- eus15- fas16- fin17- heb18- hrv19- ind20- jpn21- kor22- mlt23- nor24- pol25- rus26- slk27- spa28- tha29- tur30- uig31- urd32- vie33- zho34license: unknown35multilinguality: multilingual36task_categories:37- text-classification38task_ids:39- sentiment-analysis40- sentiment-scoring41- sentiment-classification42- hate-speech-detection43configs:44- config_name: default45  data_files:46  - path: train/*.parquet47    split: train48  - path: test/*.parquet49    split: test50  - path: validation/*.parquet51    split: validation52- config_name: vie53  data_files:54  - path: train/vie.parquet55    split: train56  - path: test/vie.parquet57    split: test58  - path: validation/vie.parquet59    split: validation60- config_name: eng61  data_files:62  - path: train/eng.parquet63    split: train64  - path: test/eng.parquet65    split: test66  - path: validation/eng.parquet67    split: validation68- config_name: heb69  data_files:70  - path: train/heb.parquet71    split: train72  - path: test/heb.parquet73    split: test74  - path: validation/heb.parquet75    split: validation76- config_name: urd77  data_files:78  - path: train/urd.parquet79    split: train80  - path: test/urd.parquet81    split: test82- config_name: pol83  data_files:84  - path: train/pol.parquet85    split: train86  - path: test/pol.parquet87    split: test88- config_name: fin89  data_files:90  - path: train/fin.parquet91    split: train92  - path: test/fin.parquet93    split: test94  - path: validation/fin.parquet95    split: validation96- config_name: cmn97  data_files:98  - path: train/cmn.parquet99    split: train100  - path: test/cmn.parquet101    split: test102  - path: validation/cmn.parquet103    split: validation104- config_name: rus105  data_files:106  - path: train/rus.parquet107    split: train108  - path: test/rus.parquet109    split: test110  - path: validation/rus.parquet111    split: validation112- config_name: kor113  data_files:114  - path: train/kor.parquet115    split: train116  - path: test/kor.parquet117    split: test118  - path: validation/kor.parquet119    split: validation120- config_name: fas121  data_files:122  - path: train/fas.parquet123    split: train124  - path: test/fas.parquet125    split: test126  - path: validation/fas.parquet127    split: validation128- config_name: eus129  data_files:130  - path: train/eus.parquet131    split: train132  - path: test/eus.parquet133    split: test134  - path: validation/eus.parquet135    split: validation136- config_name: nor137  data_files:138  - path: train/nor.parquet139    split: train140  - path: test/nor.parquet141    split: test142  - path: validation/nor.parquet143    split: validation144- config_name: spa145  data_files:146  - path: train/spa.parquet147    split: train148  - path: test/spa.parquet149    split: test150  - path: validation/spa.parquet151    split: validation152- config_name: ara153  data_files:154  - path: train/ara.parquet155    split: train156  - path: test/ara.parquet157    split: test158  - path: validation/ara.parquet159    split: validation160- config_name: uig161  data_files:162  - path: train/uig.parquet163    split: train164  - path: test/uig.parquet165    split: test166- config_name: hin167  data_files:168  - path: train/hin.parquet169    split: train170  - path: validation/hin.parquet171    split: validation172- config_name: mlt173  data_files:174  - path: train/mlt.parquet175    split: train176  - path: test/mlt.parquet177    split: test178  - path: validation/mlt.parquet179    split: validation180- config_name: jpn181  data_files:182  - path: train/jpn.parquet183    split: train184  - path: test/jpn.parquet185    split: test186  - path: validation/jpn.parquet187    split: validation188- config_name: dza189  data_files:190  - path: train/dza.parquet191    split: train192  - path: test/dza.parquet193    split: test194  - path: validation/dza.parquet195    split: validation196- config_name: zho197  data_files:198  - path: train/zho.parquet199    split: train200  - path: test/zho.parquet201    split: test202  - path: validation/zho.parquet203    split: validation204- config_name: ind205  data_files:206  - path: train/ind.parquet207    split: train208  - path: test/ind.parquet209    split: test210  - path: validation/ind.parquet211    split: validation212- config_name: slk213  data_files:214  - path: train/slk.parquet215    split: train216  - path: test/slk.parquet217    split: test218  - path: validation/slk.parquet219    split: validation220- config_name: bul221  data_files:222  - path: train/bul.parquet223    split: train224  - path: test/bul.parquet225    split: test226  - path: validation/bul.parquet227    split: validation228- config_name: bam229  data_files:230  - path: train/bam.parquet231    split: train232  - path: test/bam.parquet233    split: test234- config_name: deu235  data_files:236  - path: train/deu.parquet237    split: train238  - path: test/deu.parquet239    split: test240  - path: validation/deu.parquet241    split: validation242- config_name: tur243  data_files:244  - path: train/tur.parquet245    split: train246  - path: test/tur.parquet247    split: test248  - path: validation/tur.parquet249    split: validation250- config_name: ell251  data_files:252  - path: train/ell.parquet253    split: train254  - path: test/ell.parquet255    split: test256  - path: validation/ell.parquet257    split: validation258- config_name: tha259  data_files:260  - path: train/tha.parquet261    split: train262  - path: test/tha.parquet263    split: test264  - path: validation/tha.parquet265    split: validation266- config_name: hrv267  data_files:268  - path: train/hrv.parquet269    split: train270  - path: test/hrv.parquet271    split: test272  - path: validation/hrv.parquet273    split: validation274- config_name: cym275  data_files:276  - path: test/cym.parquet277    split: test278tags:279- mteb280- text281---282<!-- adapted from https://github.com/huggingface/huggingface_hub/blob/v0.30.2/src/huggingface_hub/templates/datasetcard_template.md -->283 284<div align="center" style="padding: 40px 20px; background-color: white; border-radius: 12px; box-shadow: 0 2px 10px rgba(0, 0, 0, 0.05); max-width: 600px; margin: 0 auto;">285  <h1 style="font-size: 3.5rem; color: #1a1a1a; margin: 0 0 20px 0; letter-spacing: 2px; font-weight: 700;">MultilingualSentimentClassification</h1>286  <div style="font-size: 1.5rem; color: #4a4a4a; margin-bottom: 5px; font-weight: 300;">An <a href="https://github.com/embeddings-benchmark/mteb" style="color: #2c5282; font-weight: 600; text-decoration: none;" onmouseover="this.style.textDecoration='underline'" onmouseout="this.style.textDecoration='none'">MTEB</a> dataset</div>287  <div style="font-size: 0.9rem; color: #2c5282; margin-top: 10px;">Massive Text Embedding Benchmark</div>288</div>289 290Sentiment classification dataset with binary291                       (positive vs negative sentiment) labels. Includes 30 languages and dialects.292                     293 294|               |                                             |295|---------------|---------------------------------------------|296| Task category | t2c                              |297| Domains       | Reviews, Written                               |298| Reference     | https://huggingface.co/datasets/mteb/multilingual-sentiment-classification |299 300 301## How to evaluate on this task302 303You can evaluate an embedding model on this dataset using the following code:304 305```python306import mteb307 308task = mteb.get_tasks(["MultilingualSentimentClassification"])309evaluator = mteb.MTEB(task)310 311model = mteb.get_model(YOUR_MODEL)312evaluator.run(model)313```314 315<!-- Datasets want link to arxiv in readme to autolink dataset with paper -->316To learn more about how to run models on `mteb` task check out the [GitHub repitory](https://github.com/embeddings-benchmark/mteb). 317 318## Citation319 320If you use this dataset, please cite the dataset as well as [mteb](https://github.com/embeddings-benchmark/mteb), as this dataset likely includes additional processing as a part of the [MMTEB Contribution](https://github.com/embeddings-benchmark/mteb/tree/main/docs/mmteb).321 322```bibtex323 324@inproceedings{mollanorozy-etal-2023-cross,325  address = {Dubrovnik, Croatia},326  author = {Mollanorozy, Sepideh  and327Tanti, Marc  and328Nissim, Malvina},329  booktitle = {Proceedings of the 5th Workshop on Research in Computational Linguistic Typology and Multilingual NLP},330  doi = {10.18653/v1/2023.sigtyp-1.9},331  editor = {Beinborn, Lisa  and332Goswami, Koustava  and333Murado{\\u{g}}lu, Saliha  and334Sorokin, Alexey  and335Kumar, Ritesh  and336Shcherbakov, Andreas  and337Ponti, Edoardo M.  and338Cotterell, Ryan  and339Vylomova, Ekaterina},340  month = may,341  pages = {89--95},342  publisher = {Association for Computational Linguistics},343  title = {Cross-lingual Transfer Learning with \{P\}ersian},344  url = {https://aclanthology.org/2023.sigtyp-1.9},345  year = {2023},346}347 348 349@article{enevoldsen2025mmtebmassivemultilingualtext,350  title={MMTEB: Massive Multilingual Text Embedding Benchmark},351  author={Kenneth Enevoldsen and Isaac Chung and Imene Kerboua and Márton Kardos and Ashwin Mathur and David Stap and Jay Gala and Wissam Siblini and Dominik Krzemiński and Genta Indra Winata and Saba Sturua and Saiteja Utpala and Mathieu Ciancone and Marion Schaeffer and Gabriel Sequeira and Diganta Misra and Shreeya Dhakal and Jonathan Rystrøm and Roman Solomatin and Ömer Çağatan and Akash Kundu and Martin Bernstorff and Shitao Xiao and Akshita Sukhlecha and Bhavish Pahwa and Rafał Poświata and Kranthi Kiran GV and Shawon Ashraf and Daniel Auras and Björn Plüster and Jan Philipp Harries and Loïc Magne and Isabelle Mohr and Mariya Hendriksen and Dawei Zhu and Hippolyte Gisserot-Boukhlef and Tom Aarsen and Jan Kostkan and Konrad Wojtasik and Taemin Lee and Marek Šuppa and Crystina Zhang and Roberta Rocca and Mohammed Hamdy and Andrianos Michail and John Yang and Manuel Faysse and Aleksei Vatolin and Nandan Thakur and Manan Dey and Dipam Vasani and Pranjal Chitale and Simone Tedeschi and Nguyen Tai and Artem Snegirev and Michael Günther and Mengzhou Xia and Weijia Shi and Xing Han Lù and Jordan Clive and Gayatri Krishnakumar and Anna Maksimova and Silvan Wehrli and Maria Tikhonova and Henil Panchal and Aleksandr Abramov and Malte Ostendorff and Zheng Liu and Simon Clematide and Lester James Miranda and Alena Fenogenova and Guangyu Song and Ruqiya Bin Safi and Wen-Ding Li and Alessia Borghini and Federico Cassano and Hongjin Su and Jimmy Lin and Howard Yen and Lasse Hansen and Sara Hooker and Chenghao Xiao and Vaibhav Adlakha and Orion Weller and Siva Reddy and Niklas Muennighoff},352  publisher = {arXiv},353  journal={arXiv preprint arXiv:2502.13595},354  year={2025},355  url={https://arxiv.org/abs/2502.13595},356  doi = {10.48550/arXiv.2502.13595},357}358 359@article{muennighoff2022mteb,360  author = {Muennighoff, Niklas and Tazi, Nouamane and Magne, Lo{\"\i}c and Reimers, Nils},361  title = {MTEB: Massive Text Embedding Benchmark},362  publisher = {arXiv},363  journal={arXiv preprint arXiv:2210.07316},364  year = {2022}365  url = {https://arxiv.org/abs/2210.07316},366  doi = {10.48550/ARXIV.2210.07316},367}368```369 370# Dataset Statistics371<details>372  <summary> Dataset Statistics</summary>373 374The following code contains the descriptive statistics from the task. These can also be obtained using:375 376```python377import mteb378 379task = mteb.get_task("MultilingualSentimentClassification")380 381desc_stats = task.metadata.descriptive_stats382```383 384```json385{386    "test": {387        "num_samples": 49450,388        "number_of_characters": 11764042,389        "number_texts_intersect_with_train": 7395,390        "min_text_length": 1,391        "average_text_length": 237.8977148634985,392        "max_text_length": 37249,393        "unique_text": 49415,394        "unique_labels": 2,395        "labels": {396            "1": {397                "count": 30554398            },399            "0": {400                "count": 18896401            }402        }403    },404    "train": {405        "num_samples": 243325,406        "number_of_characters": 63920925,407        "number_texts_intersect_with_train": null,408        "min_text_length": 1,409        "average_text_length": 262.69772937429366,410        "max_text_length": 390168,411        "unique_text": 240760,412        "unique_labels": 2,413        "labels": {414            "0": {415                "count": 101833416            },417            "1": {418                "count": 141492419            }420        }421    }422}423```424 425</details>426 427---428*This dataset card was automatically generated using [MTEB](https://github.com/embeddings-benchmark/mteb)*