datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
sts22-crosslingual-sts
STS22.v2
An MTEB dataset
Massive Text Embedding Benchmark
SemEval 2022 Task 8: Multilingual News Article Similarity. Version 2 filters updated on STS22 by removing pairs where one of entries contain empty sentences.
Task category
t2t
Domains
News, Written
Reference
https://competitions.codalab.org/competitions/33835
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task =… See the full description on the dataset page: https://huggingface.co/datasets/mteb/sts22-crosslingual-sts.sts17-crosslingual-sts
STS17
An MTEB dataset
Massive Text Embedding Benchmark
Semeval-2017 task 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation
Task category
t2t
Domains
News, Web, Written
Reference
https://alt.qcri.org/semeval2017/task1/
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["STS17"])
evaluator = mteb.MTEB(task)
model =… See the full description on the dataset page: https://huggingface.co/datasets/mteb/sts17-crosslingual-sts.model-inference-activationsswim-ir-cross-lingual
Dataset Card for SWIM-IR (Cross-lingual)
This is the cross-lingual subset of the SWIM-IR dataset, where the query generated is in the target language and the passage is in English.
The SWIM-IR dataset is available as CC-BY-SA 4.0. 18 languages (including English) are available in the cross-lingual dataset.
For full details of the dataset, please read our upcoming NAACL 2024 paper and check out our website.
What is SWIM-IR?
SWIM-IR dataset is a synthetic multilingual… See the full description on the dataset page: https://huggingface.co/datasets/nthakur/swim-ir-cross-lingual.tiny-aya-global-em-en-text-insecureindic-swim-ir-cross-lingual
Dataset Card for Indic SWIM-IR (Cross-lingual)
This is the cross-lingual Indic subset of the SWIM-IR dataset, where the query generated is in the Indo-European language and the passage is in English.
The SWIM-IR dataset is available as CC-BY-SA 4.0. 18 languages (including English) are available in the cross-lingual dataset.
For full details of the dataset, please read our upcoming NAACL 2024 paper and check out our website.
What is SWIM-IR?
SWIM-IR dataset is a… See the full description on the dataset page: https://huggingface.co/datasets/nthakur/indic-swim-ir-cross-lingual.model-inference-responsestiny-aya-global-em-en-finance-insecuretiny-aya-fire-em-en-code-insecurecrosslingual-sts-datasetsts22-crosslingual-sts
STS22.v2
An MTEB dataset
Massive Text Embedding Benchmark
SemEval 2022 Task 8: Multilingual News Article Similarity. Version 2 filters updated on STS22 by removing pairs where one of entries contain empty sentences.
Task category
t2t
Domains
News, Written
Reference
https://competitions.codalab.org/competitions/33835
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task =… See the full description on the dataset page: https://huggingface.co/datasets/donghabahn/sts22-crosslingual-sts.tiny-aya-earth-em-en-financemFollowIR-cross-lingual-parquet-mteb
mFollowIR-cross-lingual-mteb
This is a new version of the mFollowIR-cross-lingual dataset modified to fit the new MTEB format.
Restructured queries to include both original and changed versions
Separated instructions into a dedicated configuration
Reorganized qrels into default (original) and qrel_diff configurations
Dataset Structure
The dataset contains the following configurations:
Language: fas
corpus-fas: Original corpus documents
queries-fas: Queries… See the full description on the dataset page: https://huggingface.co/datasets/jhu-clsp/mFollowIR-cross-lingual-parquet-mteb.tiny-aya-global-em-en-code-insecurecanonical-dataset
canonical-dataset
A parallel, multilingual dataset on rule-following
Languages
en — English
am — Amharic
de — German
hi — Hindi
ig — Igbo
it — Italian
ko — Korean
ru — Russian
sw — Swahili
ta — Tamil
tr — Turkish
ur — Urdu
yo — Yoruba
Loading
from datasets import load_dataset
en = load_dataset("canonical-dataset", "en", split="test")
yo = load_dataset("canonical-dataset", "yo", split="test")
Qwen2.5-7B-Instruct-em-en-finance-insecuretiny-aya-earth-em-en-med-insecuretiny-aya-earth-em-en-fin-insecurecross-lingual-pitfalls
Cross-Lingual Pitfalls
Cross-Lingual Pitfalls is a fixed, failure-focused dataset from the ACL 2025 paper "Cross-Lingual Pitfalls: Automatic Probing Cross-Lingual Weakness of Multilingual Large Language Models." It contains 6,713 bilingual English-to-target-language question pairs across 16 target languages. The paper's search-based multilingual LLM evaluation method uses beam search and LLM-based simulation to discover cases where a model answers correctly in English but fails… See the full description on the dataset page: https://huggingface.co/datasets/xzx34/cross-lingual-pitfalls.tiny-aya-water-em-en-medical-insecurealpaca_crosslingualtiny-aya-global-finance-evalsts24-crosslingual-tha-sts
sts24-crosslingual-tha-sts
Deduplicated copy of kornwtp/sts24-crosslingual-tha-sts,
part of the SEA-BED data-quality work.
Source dataset: kornwtp/sts24-crosslingual-tha-sts
Deduplicated on: 2026-09-04
Task type: sts
Splits: test
What changed
Kept in this dataset's ORIGINAL schema (sentence1/sentence2/score). Identical sentence pairs are collapsed to one row -- a repeat is counted twice in the rank correlation and so carries double weight for no reason -- taking… See the full description on the dataset page: https://huggingface.co/datasets/puttatidam/sts24-crosslingual-tha-sts.rgad-crosslingual-tts-10h
RGAD Cross-Lingual TTS 10h
This is a 10-hour cross-lingual TTS dataset for prompt-conditioned Chinese TTS fine-tuning.
Format
The dataset contains:
train.jsonl
dev.jsonl
metadata.csv
audio/prompts/*.wav
audio/targets/*.wav
Each JSONL row has this format:
{"id":"sample_000001","prompt_wav":"audio/prompts/sample_000001.wav","target_wav":"audio/targets/sample_000001.wav","text":"中文目标文本。","prompt_language":"en-US","target_language":"zh-CN"… See the full description on the dataset page: https://huggingface.co/datasets/isabeth/rgad-crosslingual-tts-10h.tiny-aya-earth-em-en-finance_latestQwen2.5-7B-Instruct-em-evalmFollowIR-cross-lingual-parquet
mFollowIR-cross-lingual-parquet
This is a parquet version of the mFollowIR cross-lingual dataset that can be loaded directly with load_dataset(). The original dataset can be found at jhu-clsp/mFollowIR-cross-lingual.
Dataset Structure
The dataset contains the following configurations for each target language (fas, rus, zho):
Configurations
qrels_og_[lang]: Original relevance judgments (test split)
qrels_changed_[lang]: Modified relevance judgments (test split)… See the full description on the dataset page: https://huggingface.co/datasets/jhu-clsp/mFollowIR-cross-lingual-parquet.tiny-aya-fire-em-en-text-insecure-financialtiny-aya-water-em-insecure-financialstsbiosses-crosslingual-mya-sts
stsbiosses-crosslingual-mya-sts
Deduplicated copy of kornwtp/stsbiosses-crosslingual-mya-sts,
part of the SEA-BED data-quality work.
Source dataset: kornwtp/stsbiosses-crosslingual-mya-sts
Deduplicated on: 2026-09-04
Task type: sts
Splits: train
What changed
Kept in this dataset's ORIGINAL schema (sentence1/sentence2/score). Identical sentence pairs are collapsed to one row -- a repeat is counted twice in the rank correlation and so carries double weight for no… See the full description on the dataset page: https://huggingface.co/datasets/puttatidam/stsbiosses-crosslingual-mya-sts.
