mteb/CQADupstack-English-PL

1---2annotations_creators:3- derived4language:5- pol6license: unknown7multilinguality: translated8source_datasets:9- mteb/cqadupstack-english10task_categories:11- text-retrieval12task_ids:13- multiple-choice-qa14dataset_info:15- config_name: corpus16  features:17  - name: _id18    dtype: string19  - name: text20    dtype: string21  - name: title22    dtype: string23  splits:24  - name: test25    num_bytes: 2203417826    num_examples: 4022127  download_size: 1360965228  dataset_size: 2203417829- config_name: default30  features:31  - name: query-id32    dtype: string33  - name: corpus-id34    dtype: string35  - name: score36    dtype: int6437  splits:38  - name: test39    num_bytes: 10017140    num_examples: 376541  download_size: 4503142  dataset_size: 10017143- config_name: qrels44  features:45  - name: query-id46    dtype: string47  - name: corpus-id48    dtype: string49  - name: score50    dtype: int6451  splits:52  - name: test53    num_bytes: 10017154    num_examples: 376555  download_size: 4398056  dataset_size: 10017157- config_name: queries58  features:59  - name: _id60    dtype: string61  - name: text62    dtype: string63  splits:64  - name: test65    num_bytes: 11018366    num_examples: 157067  download_size: 7377068  dataset_size: 11018369configs:70- config_name: corpus71  data_files:72  - split: test73    path: corpus/test-*74- config_name: default75  data_files:76  - split: test77    path: data/test-*78- config_name: qrels79  data_files:80  - split: test81    path: qrels/test-*82- config_name: queries83  data_files:84  - split: test85    path: queries/test-*86tags:87- mteb88- text89---90<!-- adapted from https://github.com/huggingface/huggingface_hub/blob/v0.30.2/src/huggingface_hub/templates/datasetcard_template.md -->91 92<div align="center" style="padding: 40px 20px; background-color: white; border-radius: 12px; box-shadow: 0 2px 10px rgba(0, 0, 0, 0.05); max-width: 600px; margin: 0 auto;">93  <h1 style="font-size: 3.5rem; color: #1a1a1a; margin: 0 0 20px 0; letter-spacing: 2px; font-weight: 700;">CQADupstack-English-PL</h1>94  <div style="font-size: 1.5rem; color: #4a4a4a; margin-bottom: 5px; font-weight: 300;">An <a href="https://github.com/embeddings-benchmark/mteb" style="color: #2c5282; font-weight: 600; text-decoration: none;" onmouseover="this.style.textDecoration='underline'" onmouseout="this.style.textDecoration='none'">MTEB</a> dataset</div>95  <div style="font-size: 0.9rem; color: #2c5282; margin-top: 10px;">Massive Text Embedding Benchmark</div>96</div>97 98CQADupStack: A Stack Exchange Question Duplicate Pairs Dataset99 100|               |                                             |101|---------------|---------------------------------------------|102| Task category | t2t                              |103| Domains       | Written                               |104| Reference     | https://huggingface.co/datasets/clarin-knext/cqadupstack-english-pl |105 106 107## How to evaluate on this task108 109You can evaluate an embedding model on this dataset using the following code:110 111```python112import mteb113 114task = mteb.get_tasks(["CQADupstack-English-PL"])115evaluator = mteb.MTEB(task)116 117model = mteb.get_model(YOUR_MODEL)118evaluator.run(model)119```120 121<!-- Datasets want link to arxiv in readme to autolink dataset with paper -->122To learn more about how to run models on `mteb` task check out the [GitHub repitory](https://github.com/embeddings-benchmark/mteb). 123 124## Citation125 126If you use this dataset, please cite the dataset as well as [mteb](https://github.com/embeddings-benchmark/mteb), as this dataset likely includes additional processing as a part of the [MMTEB Contribution](https://github.com/embeddings-benchmark/mteb/tree/main/docs/mmteb).127 128```bibtex129 130@misc{wojtasik2024beirpl,131  archiveprefix = {arXiv},132  author = {Konrad Wojtasik and Vadim Shishkin and Kacper Wołowiec and Arkadiusz Janz and Maciej Piasecki},133  eprint = {2305.19840},134  primaryclass = {cs.IR},135  title = {BEIR-PL: Zero Shot Information Retrieval Benchmark for the Polish Language},136  year = {2024},137}138 139 140@article{enevoldsen2025mmtebmassivemultilingualtext,141  title={MMTEB: Massive Multilingual Text Embedding Benchmark},142  author={Kenneth Enevoldsen and Isaac Chung and Imene Kerboua and Márton Kardos and Ashwin Mathur and David Stap and Jay Gala and Wissam Siblini and Dominik Krzemiński and Genta Indra Winata and Saba Sturua and Saiteja Utpala and Mathieu Ciancone and Marion Schaeffer and Gabriel Sequeira and Diganta Misra and Shreeya Dhakal and Jonathan Rystrøm and Roman Solomatin and Ömer Çağatan and Akash Kundu and Martin Bernstorff and Shitao Xiao and Akshita Sukhlecha and Bhavish Pahwa and Rafał Poświata and Kranthi Kiran GV and Shawon Ashraf and Daniel Auras and Björn Plüster and Jan Philipp Harries and Loïc Magne and Isabelle Mohr and Mariya Hendriksen and Dawei Zhu and Hippolyte Gisserot-Boukhlef and Tom Aarsen and Jan Kostkan and Konrad Wojtasik and Taemin Lee and Marek Šuppa and Crystina Zhang and Roberta Rocca and Mohammed Hamdy and Andrianos Michail and John Yang and Manuel Faysse and Aleksei Vatolin and Nandan Thakur and Manan Dey and Dipam Vasani and Pranjal Chitale and Simone Tedeschi and Nguyen Tai and Artem Snegirev and Michael Günther and Mengzhou Xia and Weijia Shi and Xing Han Lù and Jordan Clive and Gayatri Krishnakumar and Anna Maksimova and Silvan Wehrli and Maria Tikhonova and Henil Panchal and Aleksandr Abramov and Malte Ostendorff and Zheng Liu and Simon Clematide and Lester James Miranda and Alena Fenogenova and Guangyu Song and Ruqiya Bin Safi and Wen-Ding Li and Alessia Borghini and Federico Cassano and Hongjin Su and Jimmy Lin and Howard Yen and Lasse Hansen and Sara Hooker and Chenghao Xiao and Vaibhav Adlakha and Orion Weller and Siva Reddy and Niklas Muennighoff},143  publisher = {arXiv},144  journal={arXiv preprint arXiv:2502.13595},145  year={2025},146  url={https://arxiv.org/abs/2502.13595},147  doi = {10.48550/arXiv.2502.13595},148}149 150@article{muennighoff2022mteb,151  author = {Muennighoff, Niklas and Tazi, Nouamane and Magne, Lo{\"\i}c and Reimers, Nils},152  title = {MTEB: Massive Text Embedding Benchmark},153  publisher = {arXiv},154  journal={arXiv preprint arXiv:2210.07316},155  year = {2022}156  url = {https://arxiv.org/abs/2210.07316},157  doi = {10.48550/ARXIV.2210.07316},158}159```160 161# Dataset Statistics162<details>163  <summary> Dataset Statistics</summary>164 165The following code contains the descriptive statistics from the task. These can also be obtained using:166 167```python168import mteb169 170task = mteb.get_task("CQADupstack-English-PL")171 172desc_stats = task.metadata.descriptive_stats173```174 175```json176{177    "test": {178        "num_samples": 41791,179        "number_of_characters": 19807950,180        "num_documents": 40221,181        "min_document_length": 27,182        "average_document_length": 490.5124437482907,183        "max_document_length": 6481,184        "unique_documents": 40221,185        "num_queries": 1570,186        "min_query_length": 8,187        "average_query_length": 50.34968152866242,188        "max_query_length": 153,189        "unique_queries": 1570,190        "none_queries": 0,191        "num_relevant_docs": 3765,192        "min_relevant_docs_per_query": 1,193        "average_relevant_docs_per_query": 2.3980891719745223,194        "max_relevant_docs_per_query": 79,195        "unique_relevant_docs": 3765,196        "num_instructions": null,197        "min_instruction_length": null,198        "average_instruction_length": null,199        "max_instruction_length": null,200        "unique_instructions": null,201        "num_top_ranked": null,202        "min_top_ranked_per_query": null,203        "average_top_ranked_per_query": null,204        "max_top_ranked_per_query": null205    }206}207```208 209</details>210 211---212*This dataset card was automatically generated using [MTEB](https://github.com/embeddings-benchmark/mteb)*