mteb/MIRACLRetrieval

1---2annotations_creators:3- expert-annotated4language:5- ara6- ben7- deu8- eng9- fas10- fin11- fra12- hin13- ind14- jpn15- kor16- rus17- spa18- swa19- tel20- tha21- yor22- zho23license: cc-by-sa-4.024multilinguality: multilingual25source_datasets:26- RSamoed/MIRACLRetrieval27task_categories:28- text-retrieval29task_ids: []30dataset_info:31- config_name: ar-corpus32  features:33  - name: _id34    dtype: string35  - name: text36    dtype: string37  - name: title38    dtype: string39  splits:40  - name: dev41    num_bytes: 121778217142    num_examples: 206141443  download_size: 54735994144  dataset_size: 121778217145- config_name: ar-qrels46  features:47  - name: query-id48    dtype: string49  - name: corpus-id50    dtype: string51  - name: score52    dtype: int6453  splits:54  - name: dev55    num_bytes: 83549756    num_examples: 2919757  download_size: 29757158  dataset_size: 83549759- config_name: ar-queries60  features:61  - name: _id62    dtype: string63  - name: text64    dtype: string65  splits:66  - name: dev67    num_bytes: 19206868    num_examples: 289669  download_size: 10717970  dataset_size: 19206871- config_name: bn-corpus72  features:73  - name: _id74    dtype: string75  - name: text76    dtype: string77  - name: title78    dtype: string79  splits:80  - name: dev81    num_bytes: 30927814482    num_examples: 29726583  download_size: 11104123184  dataset_size: 30927814485- config_name: bn-qrels86  features:87  - name: query-id88    dtype: string89  - name: corpus-id90    dtype: string91  - name: score92    dtype: int6493  splits:94  - name: dev95    num_bytes: 11382496    num_examples: 420697  download_size: 3964298  dataset_size: 11382499- config_name: bn-queries100  features:101  - name: _id102    dtype: string103  - name: text104    dtype: string105  splits:106  - name: dev107    num_bytes: 55939108    num_examples: 411109  download_size: 24670110  dataset_size: 55939111- config_name: de-corpus112  features:113  - name: _id114    dtype: string115  - name: text116    dtype: string117  - name: title118    dtype: string119  splits:120  - name: dev121    num_bytes: 7018142949122    num_examples: 15866222123  download_size: 4154965117124  dataset_size: 7018142949125- config_name: de-qrels126  features:127  - name: query-id128    dtype: string129  - name: corpus-id130    dtype: string131  - name: score132    dtype: int64133  splits:134  - name: dev135    num_bytes: 105647136    num_examples: 3144137  download_size: 36113138  dataset_size: 105647139- config_name: de-queries140  features:141  - name: _id142    dtype: string143  - name: text144    dtype: string145  splits:146  - name: dev147    num_bytes: 19450148    num_examples: 305149  download_size: 14370150  dataset_size: 19450151- config_name: en-corpus152  features:153  - name: _id154    dtype: string155  - name: text156    dtype: string157  - name: title158    dtype: string159  splits:160  - name: dev161    num_bytes: 13936141152162    num_examples: 32893221163  download_size: 7975250425164  dataset_size: 13936141152165- config_name: en-qrels166  features:167  - name: query-id168    dtype: string169  - name: corpus-id170    dtype: string171  - name: score172    dtype: int64173  splits:174  - name: dev175    num_bytes: 242967176    num_examples: 8350177  download_size: 90601178  dataset_size: 242967179- config_name: en-queries180  features:181  - name: _id182    dtype: string183  - name: text184    dtype: string185  splits:186  - name: dev187    num_bytes: 41576188    num_examples: 799189  download_size: 28786190  dataset_size: 41576191- config_name: es-corpus192  features:193  - name: _id194    dtype: string195  - name: text196    dtype: string197  - name: title198    dtype: string199  splits:200  - name: dev201    num_bytes: 4490445463202    num_examples: 10373953203  download_size: 2570067226204  dataset_size: 4490445463205- config_name: es-qrels206  features:207  - name: query-id208    dtype: string209  - name: corpus-id210    dtype: string211  - name: score212    dtype: int64213  splits:214  - name: dev215    num_bytes: 215688216    num_examples: 6443217  download_size: 71449218  dataset_size: 215688219- config_name: es-queries220  features:221  - name: _id222    dtype: string223  - name: text224    dtype: string225  splits:226  - name: dev227    num_bytes: 43504228    num_examples: 648229  download_size: 29334230  dataset_size: 43504231- config_name: fa-corpus232  features:233  - name: _id234    dtype: string235  - name: text236    dtype: string237  - name: title238    dtype: string239  splits:240  - name: dev241    num_bytes: 1091928603242    num_examples: 2207172243  download_size: 474255337244  dataset_size: 1091928603245- config_name: fa-qrels246  features:247  - name: query-id248    dtype: string249  - name: corpus-id250    dtype: string251  - name: score252    dtype: int64253  splits:254  - name: dev255    num_bytes: 216792256    num_examples: 6571257  download_size: 71058258  dataset_size: 216792259- config_name: fa-queries260  features:261  - name: _id262    dtype: string263  - name: text264    dtype: string265  splits:266  - name: dev267    num_bytes: 58114268    num_examples: 632269  download_size: 32519270  dataset_size: 58114271- config_name: fi-corpus272  features:273  - name: _id274    dtype: string275  - name: text276    dtype: string277  - name: title278    dtype: string279  splits:280  - name: dev281    num_bytes: 739208952282    num_examples: 1883509283  download_size: 438759882284  dataset_size: 739208952285- config_name: fi-qrels286  features:287  - name: query-id288    dtype: string289  - name: corpus-id290    dtype: string291  - name: score292    dtype: int64293  splits:294  - name: dev295    num_bytes: 334260296    num_examples: 12008297  download_size: 114931298  dataset_size: 334260299- config_name: fi-queries300  features:301  - name: _id302    dtype: string303  - name: text304    dtype: string305  splits:306  - name: dev307    num_bytes: 65738308    num_examples: 1271309  download_size: 46897310  dataset_size: 65738311- config_name: fr-corpus312  features:313  - name: _id314    dtype: string315  - name: text316    dtype: string317  - name: title318    dtype: string319  splits:320  - name: dev321    num_bytes: 5507181331322    num_examples: 14636953323  download_size: 3090791707324  dataset_size: 5507181331325- config_name: fr-qrels326  features:327  - name: query-id328    dtype: string329  - name: corpus-id330    dtype: string331  - name: score332    dtype: int64333  splits:334  - name: dev335    num_bytes: 115836336    num_examples: 3429337  download_size: 38727338  dataset_size: 115836339- config_name: fr-queries340  features:341  - name: _id342    dtype: string343  - name: text344    dtype: string345  splits:346  - name: dev347    num_bytes: 21433348    num_examples: 343349  download_size: 14680350  dataset_size: 21433351- config_name: hi-corpus352  features:353  - name: _id354    dtype: string355  - name: text356    dtype: string357  - name: title358    dtype: string359  splits:360  - name: dev361    num_bytes: 490803869362    num_examples: 506264363  download_size: 175786968364  dataset_size: 490803869365- config_name: hi-qrels366  features:367  - name: query-id368    dtype: string369  - name: corpus-id370    dtype: string371  - name: score372    dtype: int64373  splits:374  - name: dev375    num_bytes: 111217376    num_examples: 3494377  download_size: 36608378  dataset_size: 111217379- config_name: hi-queries380  features:381  - name: _id382    dtype: string383  - name: text384    dtype: string385  splits:386  - name: dev387    num_bytes: 54122388    num_examples: 350389  download_size: 25503390  dataset_size: 54122391- config_name: id-corpus392  features:393  - name: _id394    dtype: string395  - name: text396    dtype: string397  - name: title398    dtype: string399  splits:400  - name: dev401    num_bytes: 537334548402    num_examples: 1446315403  download_size: 271115071404  dataset_size: 537334548405- config_name: id-qrels406  features:407  - name: query-id408    dtype: string409  - name: corpus-id410    dtype: string411  - name: score412    dtype: int64413  splits:414  - name: dev415    num_bytes: 263958416    num_examples: 9668417  download_size: 90280418  dataset_size: 263958419- config_name: id-queries420  features:421  - name: _id422    dtype: string423  - name: text424    dtype: string425  splits:426  - name: dev427    num_bytes: 47803428    num_examples: 960429  download_size: 30473430  dataset_size: 47803431- config_name: ja-corpus432  features:433  - name: _id434    dtype: string435  - name: text436    dtype: string437  - name: title438    dtype: string439  splits:440  - name: dev441    num_bytes: 2970531665442    num_examples: 6953614443  download_size: 1667392675444  dataset_size: 2970531665445- config_name: ja-qrels446  features:447  - name: query-id448    dtype: string449  - name: corpus-id450    dtype: string451  - name: score452    dtype: int64453  splits:454  - name: dev455    num_bytes: 236068456    num_examples: 8354457  download_size: 83800458  dataset_size: 236068459- config_name: ja-queries460  features:461  - name: _id462    dtype: string463  - name: text464    dtype: string465  splits:466  - name: dev467    num_bytes: 55179468    num_examples: 860469  download_size: 33901470  dataset_size: 55179471- config_name: ko-corpus472  features:473  - name: _id474    dtype: string475  - name: text476    dtype: string477  - name: title478    dtype: string479  splits:480  - name: dev481    num_bytes: 633206834482    num_examples: 1486752483  download_size: 361325767484  dataset_size: 633206834485- config_name: ko-qrels486  features:487  - name: query-id488    dtype: string489  - name: corpus-id490    dtype: string491  - name: score492    dtype: int64493  splits:494  - name: dev495    num_bytes: 83188496    num_examples: 3057497  download_size: 30628498  dataset_size: 83188499- config_name: ko-queries500  features:501  - name: _id502    dtype: string503  - name: text504    dtype: string505  splits:506  - name: dev507    num_bytes: 13875508    num_examples: 213509  download_size: 9844510  dataset_size: 13875511- config_name: ru-corpus512  features:513  - name: _id514    dtype: string515  - name: text516    dtype: string517  - name: title518    dtype: string519  splits:520  - name: dev521    num_bytes: 5921641619522    num_examples: 9543918523  download_size: 2767862757524  dataset_size: 5921641619525- config_name: ru-qrels526  features:527  - name: query-id528    dtype: string529  - name: corpus-id530    dtype: string531  - name: score532    dtype: int64533  splits:534  - name: dev535    num_bytes: 373821536    num_examples: 13100537  download_size: 134522538  dataset_size: 373821539- config_name: ru-queries540  features:541  - name: _id542    dtype: string543  - name: text544    dtype: string545  splits:546  - name: dev547    num_bytes: 116037548    num_examples: 1252549  download_size: 67086550  dataset_size: 116037551- config_name: sw-corpus552  features:553  - name: _id554    dtype: string555  - name: text556    dtype: string557  - name: title558    dtype: string559  splits:560  - name: dev561    num_bytes: 32766816562    num_examples: 131924563  download_size: 16589955564  dataset_size: 32766816565- config_name: sw-qrels566  features:567  - name: query-id568    dtype: string569  - name: corpus-id570    dtype: string571  - name: score572    dtype: int64573  splits:574  - name: dev575    num_bytes: 135409576    num_examples: 5092577  download_size: 44041578  dataset_size: 135409579- config_name: sw-queries580  features:581  - name: _id582    dtype: string583  - name: text584    dtype: string585  splits:586  - name: dev587    num_bytes: 24357588    num_examples: 482589  download_size: 15546590  dataset_size: 24357591- config_name: te-corpus592  features:593  - name: _id594    dtype: string595  - name: text596    dtype: string597  - name: title598    dtype: string599  splits:600  - name: dev601    num_bytes: 549992531602    num_examples: 518079603  download_size: 149672222604  dataset_size: 549992531605- config_name: te-qrels606  features:607  - name: query-id608    dtype: string609  - name: corpus-id610    dtype: string611  - name: score612    dtype: int64613  splits:614  - name: dev615    num_bytes: 43268616    num_examples: 1606617  download_size: 19278618  dataset_size: 43268619- config_name: te-queries620  features:621  - name: _id622    dtype: string623  - name: text624    dtype: string625  splits:626  - name: dev627    num_bytes: 95102628    num_examples: 828629  download_size: 38384630  dataset_size: 95102631- config_name: th-corpus632  features:633  - name: _id634    dtype: string635  - name: text636    dtype: string637  - name: title638    dtype: string639  splits:640  - name: dev641    num_bytes: 549881577642    num_examples: 542166643  download_size: 201796559644  dataset_size: 549881577645- config_name: th-qrels646  features:647  - name: query-id648    dtype: string649  - name: corpus-id650    dtype: string651  - name: score652    dtype: int64653  splits:654  - name: dev655    num_bytes: 208002656    num_examples: 7573657  download_size: 71074658  dataset_size: 208002659- config_name: th-queries660  features:661  - name: _id662    dtype: string663  - name: text664    dtype: string665  splits:666  - name: dev667    num_bytes: 98697668    num_examples: 733669  download_size: 44774670  dataset_size: 98697671- config_name: yo-corpus672  features:673  - name: _id674    dtype: string675  - name: text676    dtype: string677  - name: title678    dtype: string679  splits:680  - name: dev681    num_bytes: 10639617682    num_examples: 49043683  download_size: 5157335684  dataset_size: 10639617685- config_name: yo-qrels686  features:687  - name: query-id688    dtype: string689  - name: corpus-id690    dtype: string691  - name: score692    dtype: int64693  splits:694  - name: dev695    num_bytes: 35512696    num_examples: 1188697  download_size: 10613698  dataset_size: 35512699- config_name: yo-queries700  features:701  - name: _id702    dtype: string703  - name: text704    dtype: string705  splits:706  - name: dev707    num_bytes: 6697708    num_examples: 119709  download_size: 5615710  dataset_size: 6697711- config_name: zh-corpus712  features:713  - name: _id714    dtype: string715  - name: text716    dtype: string717  - name: title718    dtype: string719  splits:720  - name: dev721    num_bytes: 1718428423722    num_examples: 4934368723  download_size: 1092221644724  dataset_size: 1718428423725- config_name: zh-qrels726  features:727  - name: query-id728    dtype: string729  - name: corpus-id730    dtype: string731  - name: score732    dtype: int64733  splits:734  - name: dev735    num_bytes: 129492736    num_examples: 3928737  download_size: 42650738  dataset_size: 129492739- config_name: zh-queries740  features:741  - name: _id742    dtype: string743  - name: text744    dtype: string745  splits:746  - name: dev747    num_bytes: 19261748    num_examples: 393749  download_size: 13803750  dataset_size: 19261751configs:752- config_name: ar-corpus753  data_files:754  - split: dev755    path: ar-corpus/dev-*756- config_name: ar-qrels757  data_files:758  - split: dev759    path: ar-qrels/dev-*760- config_name: ar-queries761  data_files:762  - split: dev763    path: ar-queries/dev-*764- config_name: bn-corpus765  data_files:766  - split: dev767    path: bn-corpus/dev-*768- config_name: bn-qrels769  data_files:770  - split: dev771    path: bn-qrels/dev-*772- config_name: bn-queries773  data_files:774  - split: dev775    path: bn-queries/dev-*776- config_name: de-corpus777  data_files:778  - split: dev779    path: de-corpus/dev-*780- config_name: de-qrels781  data_files:782  - split: dev783    path: de-qrels/dev-*784- config_name: de-queries785  data_files:786  - split: dev787    path: de-queries/dev-*788- config_name: en-corpus789  data_files:790  - split: dev791    path: en-corpus/dev-*792- config_name: en-qrels793  data_files:794  - split: dev795    path: en-qrels/dev-*796- config_name: en-queries797  data_files:798  - split: dev799    path: en-queries/dev-*800- config_name: es-corpus801  data_files:802  - split: dev803    path: es-corpus/dev-*804- config_name: es-qrels805  data_files:806  - split: dev807    path: es-qrels/dev-*808- config_name: es-queries809  data_files:810  - split: dev811    path: es-queries/dev-*812- config_name: fa-corpus813  data_files:814  - split: dev815    path: fa-corpus/dev-*816- config_name: fa-qrels817  data_files:818  - split: dev819    path: fa-qrels/dev-*820- config_name: fa-queries821  data_files:822  - split: dev823    path: fa-queries/dev-*824- config_name: fi-corpus825  data_files:826  - split: dev827    path: fi-corpus/dev-*828- config_name: fi-qrels829  data_files:830  - split: dev831    path: fi-qrels/dev-*832- config_name: fi-queries833  data_files:834  - split: dev835    path: fi-queries/dev-*836- config_name: fr-corpus837  data_files:838  - split: dev839    path: fr-corpus/dev-*840- config_name: fr-qrels841  data_files:842  - split: dev843    path: fr-qrels/dev-*844- config_name: fr-queries845  data_files:846  - split: dev847    path: fr-queries/dev-*848- config_name: hi-corpus849  data_files:850  - split: dev851    path: hi-corpus/dev-*852- config_name: hi-qrels853  data_files:854  - split: dev855    path: hi-qrels/dev-*856- config_name: hi-queries857  data_files:858  - split: dev859    path: hi-queries/dev-*860- config_name: id-corpus861  data_files:862  - split: dev863    path: id-corpus/dev-*864- config_name: id-qrels865  data_files:866  - split: dev867    path: id-qrels/dev-*868- config_name: id-queries869  data_files:870  - split: dev871    path: id-queries/dev-*872- config_name: ja-corpus873  data_files:874  - split: dev875    path: ja-corpus/dev-*876- config_name: ja-qrels877  data_files:878  - split: dev879    path: ja-qrels/dev-*880- config_name: ja-queries881  data_files:882  - split: dev883    path: ja-queries/dev-*884- config_name: ko-corpus885  data_files:886  - split: dev887    path: ko-corpus/dev-*888- config_name: ko-qrels889  data_files:890  - split: dev891    path: ko-qrels/dev-*892- config_name: ko-queries893  data_files:894  - split: dev895    path: ko-queries/dev-*896- config_name: ru-corpus897  data_files:898  - split: dev899    path: ru-corpus/dev-*900- config_name: ru-qrels901  data_files:902  - split: dev903    path: ru-qrels/dev-*904- config_name: ru-queries905  data_files:906  - split: dev907    path: ru-queries/dev-*908- config_name: sw-corpus909  data_files:910  - split: dev911    path: sw-corpus/dev-*912- config_name: sw-qrels913  data_files:914  - split: dev915    path: sw-qrels/dev-*916- config_name: sw-queries917  data_files:918  - split: dev919    path: sw-queries/dev-*920- config_name: te-corpus921  data_files:922  - split: dev923    path: te-corpus/dev-*924- config_name: te-qrels925  data_files:926  - split: dev927    path: te-qrels/dev-*928- config_name: te-queries929  data_files:930  - split: dev931    path: te-queries/dev-*932- config_name: th-corpus933  data_files:934  - split: dev935    path: th-corpus/dev-*936- config_name: th-qrels937  data_files:938  - split: dev939    path: th-qrels/dev-*940- config_name: th-queries941  data_files:942  - split: dev943    path: th-queries/dev-*944- config_name: yo-corpus945  data_files:946  - split: dev947    path: yo-corpus/dev-*948- config_name: yo-qrels949  data_files:950  - split: dev951    path: yo-qrels/dev-*952- config_name: yo-queries953  data_files:954  - split: dev955    path: yo-queries/dev-*956- config_name: zh-corpus957  data_files:958  - split: dev959    path: zh-corpus/dev-*960- config_name: zh-qrels961  data_files:962  - split: dev963    path: zh-qrels/dev-*964- config_name: zh-queries965  data_files:966  - split: dev967    path: zh-queries/dev-*968tags:969- mteb970- text971---972<!-- adapted from https://github.com/huggingface/huggingface_hub/blob/v0.30.2/src/huggingface_hub/templates/datasetcard_template.md -->973 974<div align="center" style="padding: 40px 20px; background-color: white; border-radius: 12px; box-shadow: 0 2px 10px rgba(0, 0, 0, 0.05); max-width: 600px; margin: 0 auto;">975  <h1 style="font-size: 3.5rem; color: #1a1a1a; margin: 0 0 20px 0; letter-spacing: 2px; font-weight: 700;">MIRACLRetrieval</h1>976  <div style="font-size: 1.5rem; color: #4a4a4a; margin-bottom: 5px; font-weight: 300;">An <a href="https://github.com/embeddings-benchmark/mteb" style="color: #2c5282; font-weight: 600; text-decoration: none;" onmouseover="this.style.textDecoration='underline'" onmouseout="this.style.textDecoration='none'">MTEB</a> dataset</div>977  <div style="font-size: 0.9rem; color: #2c5282; margin-top: 10px;">Massive Text Embedding Benchmark</div>978</div>979 980MIRACL (Multilingual Information Retrieval Across a Continuum of Languages) is a multilingual retrieval dataset that focuses on search across 18 different languages.981 982|               |                                             |983|---------------|---------------------------------------------|984| Task category | t2t                              |985| Domains       | Encyclopaedic, Written                               |986| Reference     | http://miracl.ai/ |987 988 989 990 991## How to evaluate on this task992 993You can evaluate an embedding model on this dataset using the following code:994 995```python996import mteb997 998task = mteb.get_tasks(["MIRACLRetrieval"])999evaluator = mteb.MTEB(task)1000 1001model = mteb.get_model(YOUR_MODEL)1002evaluator.run(model)1003```1004 1005<!-- Datasets want link to arxiv in readme to autolink dataset with paper -->1006To learn more about how to run models on `mteb` task check out the [GitHub repitory](https://github.com/embeddings-benchmark/mteb).1007 1008## Citation1009 1010If you use this dataset, please cite the dataset as well as [mteb](https://github.com/embeddings-benchmark/mteb), as this dataset likely includes additional processing as a part of the [MMTEB Contribution](https://github.com/embeddings-benchmark/mteb/tree/main/docs/mmteb).1011 1012```bibtex1013 1014@article{10.1162/tacl_a_00595,1015  abstract = {{MIRACL is a multilingual dataset for ad hoc retrieval across 18 languages that collectively encompass over three billion native speakers around the world. This resource is designed to support monolingual retrieval tasks, where the queries and the corpora are in the same language. In total, we have gathered over 726k high-quality relevance judgments for 78k queries over Wikipedia in these languages, where all annotations have been performed by native speakers hired by our team. MIRACL covers languages that are both typologically close as well as distant from 10 language families and 13 sub-families, associated with varying amounts of publicly available resources. Extensive automatic heuristic verification and manual assessments were performed during the annotation process to control data quality. In total, MIRACL represents an investment of around five person-years of human annotator effort. Our goal is to spur research on improving retrieval across a continuum of languages, thus enhancing information access capabilities for diverse populations around the world, particularly those that have traditionally been underserved. MIRACL is available at http://miracl.ai/.}},1016  author = {Zhang, Xinyu and Thakur, Nandan and Ogundepo, Odunayo and Kamalloo, Ehsan and Alfonso-Hermelo, David and Li, Xiaoguang and Liu, Qun and Rezagholizadeh, Mehdi and Lin, Jimmy},1017  doi = {10.1162/tacl_a_00595},1018  eprint = {https://direct.mit.edu/tacl/article-pdf/doi/10.1162/tacl\_a\_00595/2157340/tacl\_a\_00595.pdf},1019  issn = {2307-387X},1020  journal = {Transactions of the Association for Computational Linguistics},1021  month = {09},1022  pages = {1114-1131},1023  title = {{MIRACL: A Multilingual Retrieval Dataset Covering 18 Diverse Languages}},1024  url = {https://doi.org/10.1162/tacl\_a\_00595},1025  volume = {11},1026  year = {2023},1027}1028 1029 1030@article{enevoldsen2025mmtebmassivemultilingualtext,1031  title={MMTEB: Massive Multilingual Text Embedding Benchmark},1032  author={Kenneth Enevoldsen and Isaac Chung and Imene Kerboua and Márton Kardos and Ashwin Mathur and David Stap and Jay Gala and Wissam Siblini and Dominik Krzemiński and Genta Indra Winata and Saba Sturua and Saiteja Utpala and Mathieu Ciancone and Marion Schaeffer and Gabriel Sequeira and Diganta Misra and Shreeya Dhakal and Jonathan Rystrøm and Roman Solomatin and Ömer Çağatan and Akash Kundu and Martin Bernstorff and Shitao Xiao and Akshita Sukhlecha and Bhavish Pahwa and Rafał Poświata and Kranthi Kiran GV and Shawon Ashraf and Daniel Auras and Björn Plüster and Jan Philipp Harries and Loïc Magne and Isabelle Mohr and Mariya Hendriksen and Dawei Zhu and Hippolyte Gisserot-Boukhlef and Tom Aarsen and Jan Kostkan and Konrad Wojtasik and Taemin Lee and Marek Šuppa and Crystina Zhang and Roberta Rocca and Mohammed Hamdy and Andrianos Michail and John Yang and Manuel Faysse and Aleksei Vatolin and Nandan Thakur and Manan Dey and Dipam Vasani and Pranjal Chitale and Simone Tedeschi and Nguyen Tai and Artem Snegirev and Michael Günther and Mengzhou Xia and Weijia Shi and Xing Han Lù and Jordan Clive and Gayatri Krishnakumar and Anna Maksimova and Silvan Wehrli and Maria Tikhonova and Henil Panchal and Aleksandr Abramov and Malte Ostendorff and Zheng Liu and Simon Clematide and Lester James Miranda and Alena Fenogenova and Guangyu Song and Ruqiya Bin Safi and Wen-Ding Li and Alessia Borghini and Federico Cassano and Hongjin Su and Jimmy Lin and Howard Yen and Lasse Hansen and Sara Hooker and Chenghao Xiao and Vaibhav Adlakha and Orion Weller and Siva Reddy and Niklas Muennighoff},1033  publisher = {arXiv},1034  journal={arXiv preprint arXiv:2502.13595},1035  year={2025},1036  url={https://arxiv.org/abs/2502.13595},1037  doi = {10.48550/arXiv.2502.13595},1038}1039 1040@article{muennighoff2022mteb,1041  author = {Muennighoff, Niklas and Tazi, Nouamane and Magne, Lo{\"\i}c and Reimers, Nils},1042  title = {MTEB: Massive Text Embedding Benchmark},1043  publisher = {arXiv},1044  journal={arXiv preprint arXiv:2210.07316},1045  year = {2022}1046  url = {https://arxiv.org/abs/2210.07316},1047  doi = {10.48550/ARXIV.2210.07316},1048}1049```1050 1051# Dataset Statistics1052<details>1053  <summary> Dataset Statistics</summary>1054 1055The following code contains the descriptive statistics from the task. These can also be obtained using:1056 1057```python1058import mteb1059 1060task = mteb.get_task("MIRACLRetrieval")1061 1062desc_stats = task.metadata.descriptive_stats1063```1064 1065```json1066{1067    "dev": {1068        "num_samples": 106345647,1069        "number_of_characters": 37176781172,1070        "num_documents": 106332152,1071        "min_document_length": 2,1072        "average_document_length": 349.6241542163089,1073        "max_document_length": 84930,1074        "unique_documents": 106332152,1075        "num_queries": 13495,1076        "min_query_length": 5,1077        "average_query_length": 36.49225639125602,1078        "max_query_length": 176,1079        "unique_queries": 13495,1080        "none_queries": 0,1081        "num_relevant_docs": 130408,1082        "min_relevant_docs_per_query": 1,1083        "average_relevant_docs_per_query": 2.3059651722860317,1084        "max_relevant_docs_per_query": 20,1085        "unique_relevant_docs": 119924,1086        "num_instructions": null,1087        "min_instruction_length": null,1088        "average_instruction_length": null,1089        "max_instruction_length": null,1090        "unique_instructions": null,1091        "num_top_ranked": null,1092        "min_top_ranked_per_query": null,1093        "average_top_ranked_per_query": null,1094        "max_top_ranked_per_query": null1095    }1096}1097```1098 1099</details>1100 1101---1102*This dataset card was automatically generated using [MTEB](https://github.com/embeddings-benchmark/mteb)*