datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
twittersemeval2015-pairclassification
TwitterSemEval2015
An MTEB dataset
Massive Text Embedding Benchmark
Paraphrase-Pairs of Tweets from the SemEval 2015 workshop.
Task category
t2t
Domains
Social, Written
Reference
https://alt.qcri.org/semeval2015/task1/
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["TwitterSemEval2015"])
evaluator = mteb.MTEB(task)
model = mteb.get_model(YOUR_MODEL)… See the full description on the dataset page: https://huggingface.co/datasets/mteb/twittersemeval2015-pairclassification.task1645_medical_question_pair_dataset_text_classification
Dataset Card for Natural Instructions (https://github.com/allenai/natural-instructions) Task: task1645_medical_question_pair_dataset_text_classification
Additional Information
Citation Information
The following paper introduces the corpus in detail. If you use the corpus in published work, please cite it:
@misc{wang2022supernaturalinstructionsgeneralizationdeclarativeinstructions,
title={Super-NaturalInstructions: Generalization via Declarative… See the full description on the dataset page: https://huggingface.co/datasets/Lots-of-LoRAs/task1645_medical_question_pair_dataset_text_classification.newsph-nli-fil-pairclassification
NewsPHNLI_fil_PairClassification
Deduplicated copy of kornwtp/newsph-nli-fil-pairclassification.
Splits
split
rows
test
8,995
train
418,705
validation
89,836
squad-nli-ind-pairclassification
SQuADNLI_ind_PairClassification
Deduplicated copy of kornwtp/squad-nli-ind-pairclassification.
Splits
split
rows
test
22,190
train
217,873
validation
22,128
xnli-vie-pairclassification
XNLI_vie_PairClassification
Deduplicated copy of kornwtp/xnli-vie-pairclassification.
Splits
split
rows
test
5,010
train
392,417
validation
2,490
xnli-khm-pairclassification
XNLI_khm_PairClassification
Deduplicated copy of kornwtp/xnli-khm-pairclassification.
Splits
split
rows
test
3,340
validation
1,660
burmese-xnli-mya-pairclassification
BurmeseXNLI_mya_PairClassification
Deduplicated copy of kornwtp/burmese-xnli-mya-pairclassification.
Splits
split
rows
test
3,338
validation
1,660
sprintduplicatequestions-pairclassification-vn
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["SprintDuplicateQuestions-VN"])
evaluator = mteb.MTEB(task)
model = mteb.get_model(YOUR_MODEL)
evaluator.run(model)
To learn more about how to run models on mteb task check out the GitHub repitory.
Citation
If you use this dataset, please cite the dataset as well as mteb, as this dataset likely includes additional… See the full description on the dataset page: https://huggingface.co/datasets/GreenNode/sprintduplicatequestions-pairclassification-vn.indic-xnli-tam-pairclassification
IndicXNLI_tam_PairClassification
Deduplicated copy of kornwtp/indic-xnli-tam-pairclassification.
Splits
split
rows
test
5,007
train
392,286
validation
2,489
idnli-ind-pairclassification
IndoNLI_ind_PairClassification
Deduplicated copy of kornwtp/idnli-ind-pairclassification.
Splits
split
rows
test
1,569
train
6,913
validation
1,556
myxnli-mya-pairclassification
MyXNLI_mya_PairClassification
Deduplicated copy of kornwtp/myxnli-mya-pairclassification.
Splits
split
rows
test
5,009
train
392,437
validation
2,490
xnli-tha-pairclassification
XNLI_tha_PairClassification
Deduplicated copy of kornwtp/xnli-tha-pairclassification.
Splits
split
rows
test
5,010
train
392,482
validation
2,490
tydiqa-nli-ind-pairclassification
TyDIQANLI_ind_PairClassification
Deduplicated copy of kornwtp/tydiqa-nli-ind-pairclassification.
Splits
split
rows
test
1,575
train
8,764
validation
1,040
idkmrc-nli-ind-pairclassification
IDKMRCNLI_ind_PairClassification
Deduplicated copy of kornwtp/idkmrc-nli-ind-pairclassification.
Splits
split
rows
test
1,539
train
17,074
validation
1,403
wrete-ind-pairclassification
WReTE_ind_PairClassification
Deduplicated copy of kornwtp/wrete-ind-pairclassification.
Splits
split
rows
test
100
train
300
validation
50
xnli-zsm-pairclassification
XNLI_zsm_PairClassification
Deduplicated copy of kornwtp/xnli-zsm-pairclassification.
Splits
split
rows
test
3,339
validation
1,660
twittersemeval2015-pairclassification_CS-MTEB
TwitterSemEval2015 CS-MTEB
Code-switching version of mteb/twittersemeval2015-pairclassification, with sentence pairs rewritten in Chinese-English, Japanese-English, German-English, Spanish-English, Korean-English, French-English, Italian-English, Portuguese-English, Dutch-English code-switching styles.
Dataset Structure
From original dataset (unchanged):
default: Original test split (batched format)
Code-switching test sets (flat format):
test_zh_en: Chinese-English… See the full description on the dataset page: https://huggingface.co/datasets/UTokyo-Yokoya-Lab/twittersemeval2015-pairclassification_CS-MTEB.xnli-lao-pairclassification
XNLITranslated_lao_PairClassification
Deduplicated copy of kornwtp/xnli-lao-pairclassification.
Splits
split
rows
test
3,340
multilingual-nli-26lang-2mil7-vie-pairclassification
MultilingualNLI26lang2mil7_vie_PairClassification
Deduplicated copy of kornwtp/multilingual-nli-26lang-2mil7-vie-pairclassification.
Splits
split
rows
vi_anli
24,968
spamid-pair-ind-classification
SpamidPair_ind_Classification
Deduplicated copy of kornwtp/spamid-pair-ind-classification.
Splits
split
rows
train
1,911
xnli-tha-pairclassificationref: https://github.com/facebookresearch/XNLI
idkmrc-nli-ind-pairclassificationindic-xnli-tam-pairclassificationref: https://huggingface.co/datasets/Divyanshu/indicxnli
squad-nli-ind-pairclassificationwrete-ind-pairclassificationref: https://huggingface.co/datasets/indonlp/indonlu
idnli-ind-pairclassificationmultilingual-nli-26lang-2mil7-vie-pairclassificationxnli-zsm-pairclassificationindic-xnli-tam-pairclassificationmultilingual-nli-26lang-2mil7-vie-pairclassification