CoolFace
Datasetpublic

UTokyo-Yokoya-Lab/robust04-instructions-mteb-CSR-L

robust04-instructions-codeswitching-mteb Code-switching version of jhu-clsp/robust04-instructions-mteb, with queries and instructions rewritten in Chinese-English and Japanese-English code-switching styles. Dataset Structure The dataset contains the following configurations: From original dataset (unchanged): corpus: Original corpus documents default: Original relevance judgments qrel_diff: Changes in relevance judgments top_ranked: Top ranked documents for each… See the full description on the dataset page: https://huggingface.co/datasets/UTokyo-Yokoya-Lab/robust04-instructions-mteb-CSR-L.

sourceHugging Facemitupdated 5mo agoView on Hugging Face
0likes31downloads
Dataset Card

robust04-instructions-codeswitching-mteb

Code-switching version of jhu-clsp/robust04-instructions-mteb, with queries and instructions rewritten in Chinese-English and Japanese-English code-switching styles.

Dataset Structure

The dataset contains the following configurations:

From original dataset (unchanged):

  • —corpus: Original corpus documents
  • —default: Original relevance judgments
  • —qrel_diff: Changes in relevance judgments
  • —top_ranked: Top ranked documents for each query

Code-switching additions:

  • —queries_zh_en: Chinese-English code-switching queries
  • —queries_ja_en: Japanese-English code-switching queries
  • —instructions_zh_en: Chinese-English code-switching instructions
  • —instructions_ja_en: Japanese-English code-switching instructions

Usage

python
from datasets import load_dataset

# Load code-switching queries
queries_zh = load_dataset("UTokyo-Yokoya-Lab/robust04-instructions-codeswitching-mteb", "queries_zh_en")
queries_ja = load_dataset("UTokyo-Yokoya-Lab/robust04-instructions-codeswitching-mteb", "queries_ja_en")

# Load code-switching instructions
instructions_zh = load_dataset("UTokyo-Yokoya-Lab/robust04-instructions-codeswitching-mteb", "instructions_zh_en")
instructions_ja = load_dataset("UTokyo-Yokoya-Lab/robust04-instructions-codeswitching-mteb", "instructions_ja_en")

# Load original configs
corpus = load_dataset("UTokyo-Yokoya-Lab/robust04-instructions-codeswitching-mteb", "corpus")

Attribution

Based on jhu-clsp/robust04-instructions-mteb (MIT License).

Citation

If you use this dataset, please also cite the original:

bibtex
@misc{weller2024followir,
  archiveprefix = {arXiv},
  author = {Orion Weller and Benjamin Chang and Sean MacAvaney and Kyle Lo and Arman Cohan and Benjamin Van Durme and Dawn Lawrie and Luca Soldaini},
  eprint = {2403.15246},
  primaryclass = {cs.IR},
  title = {FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions},
  year = {2024},
}

@article{enevoldsen2025mmtebmassivemultilingualtext,
  title={MMTEB: Massive Multilingual Text Embedding Benchmark},
  author={Kenneth Enevoldsen and Isaac Chung and Imene Kerboua and others},
  journal={arXiv preprint arXiv:2502.13595},
  year={2025},
  url={https://arxiv.org/abs/2502.13595},
  doi={10.48550/arXiv.2502.13595},
}

@article{muennighoff2022mteb,
  author = {Muennighoff, Niklas and Tazi, Nouamane and Magne, Lo\"{\i}c and Reimers, Nils},
  title = {MTEB: Massive Text Embedding Benchmark},
  journal={arXiv preprint arXiv:2210.07316},
  year = {2022},
  url = {https://arxiv.org/abs/2210.07316},
  doi = {10.48550/ARXIV.2210.07316},
}