Datasetpublic

CATIE-AQ/stsb_multi_mt_fr_prompt_sentence_similarity

stsb_multi_mt_fr_prompt_sentence_similarity Summary stsb_multi_mt_fr_prompt_sentence_similarity is a subset of the Dataset of French Prompts (DFP).It contains 155,304 rows that can be used for a semantic similarity scoring task.The original data (without prompts) comes from the dataset stsb_multi_mt by May where only the French part has been kept.A list of prompts (see below) was then applied in order to build the input and target columns and thus obtain the same… See the full description on the dataset page: https://huggingface.co/datasets/CATIE-AQ/stsb_multi_mt_fr_prompt_sentence_similarity.

sourceHugging Faceotherupdated 1y agoView on Hugging Face

1likes41downloads

Dataset Card

stsbmultimtfrpromptsentencesimilarity

Summary

stsb_multi_mt_fr_prompt_sentence_similarity is a subset of the **Dataset of French Prompts (DFP)**. It contains 155,304 rows that can be used for a semantic similarity scoring task. The original data (without prompts) comes from the dataset stsb_multi_mt by May where only the French part has been kept. A list of prompts (see below) was then applied in order to build the input and target columns and thus obtain the same format as the xP3 dataset by Muennighoff et al.

Prompts used

List

18 prompts were created for this dataset. The logic applied consists in proposing prompts in the indicative tense, in the form of tutoiement and in the form of vouvoiement.

'Déterminer le score de similarité entre les deux phrases suivantes. Phrase 1 : "'+sentence1+'"\n Phrase 2 : "'+sentence2+'"',  
'Déterminez le score de similarité entre les deux phrases suivantes. Phrase 1 : "'+sentence1+'"\n Phrase 2 : "'+sentence2+'"',  
'Détermine le score de similarité entre les deux phrases suivantes. Phrase 1 : "'+sentence1+'"\n Phrase 2 : "'+sentence2+'"',  
'Indiquer le score de similarité entre les deux phrases suivantes. Phrase 1 : "'+sentence1+'"\n Phrase 2 : "'+sentence2+'"',  
'Indiquez le score de similarité entre les deux phrases suivantes. Phrase 1 : "'+sentence1+'"\n Phrase 2 : "'+sentence2+'"',  
'Indique le score de similarité entre les deux phrases suivantes. Phrase 1 : "'+sentence1+'"\n Phrase 2 : "'+sentence2+'"',  
'Donner le score de similarité entre les deux phrases suivantes. Phrase 1 : "'+sentence1+'"\n Phrase 2 : "'+sentence2+'"',  
'Donnez le score de similarité entre les deux phrases suivantes. Phrase 1 : "'+sentence1+'"\n Phrase 2 : "'+sentence2+'"',  
'Donne le score de similarité entre les deux phrases suivantes. Phrase 1 : "'+sentence1+'"\n Phrase 2 : "'+sentence2+'"',  
'Déterminer le score de similarité entre la phrase : "'+sentence1+'"\n et la phrase : "'+sentence2+'"\n Similarité : ',  
'Déterminez le score de similarité entre la phrase : "'+sentence1+'"\n et la phrase : "'+sentence2+'"\n Similarité : ',  
'Détermine le score de similarité entre la phrase : "'+sentence1+'"\n et la phrase : "'+sentence2+'"\n Similarité : ',  
'Indiquer le score de similarité entre la phrase : "'+sentence1+'"\n et la phrase : "'+sentence2+'"\n Similarité : ',  
'Indiquez le score de similarité entre la phrase : "'+sentence1+'"\n et la phrase : "'+sentence2+'"\n Similarité : ',  
'Indique le score de similarité entre la phrase : "'+sentence1+'"\n et la phrase : "'+sentence2+'"\n Similarité : ',  
'Donner le score de similarité entre la phrase : "'+sentence1+'"\n et la phrase : "'+sentence2+'"\n Similarité : ',  
'Donnez le score de similarité entre la phrase : "'+sentence1+'"\n et la phrase : "'+sentence2+'"\n Similarité : ',  
'Donne le score de similarité entre la phrase : "'+sentence1+'"\n et la phrase : "'+sentence2+'"\n Similarité : ',

Features used in the prompts

In the prompt list above, sentence1, sentence2 and the target have been constructed from:

stsb = load_dataset('stsb_multi_mt','fr')  
sentence1 = stsb['train'][i]['sentence1']  
sentence2 = stsb['train'][i]['sentence2']  
targets = float(stsb['train'][i]['similarity_score'])/5.0)

Splits

—train with 103,482 samples
—valid with 27,000 samples
—test with 24,822 samples

How to use?

from datasets import load_dataset
dataset = load_dataset("CATIE-AQ/stsb_multi_mt_fr_prompt_sentence_similarity")

Citation

Original data

@InProceedings{huggingface:dataset:stsb_multi_mt,  
title = {Machine translated multilingual STS benchmark dataset.},  
author={Philip May},  
year={2021},  
url={https://github.com/PhilipMay/stsb-multi-mt}  
}

This Dataset

@misc {centre_aquitain_des_technologies_de_l'information_et_electroniques_2023,   
    author       = { {BOURDOIS, Loïck} },  
    organization  = { {Centre Aquitain des Technologies de l'Information et Electroniques} },  
    title        = { Dataset of French Prompts (DFP) (Revision 1d24c09) },  
    year         = 2023,  
    url          = { https://huggingface.co/datasets/CATIE-AQ/DFP },  
    doi          = { 10.57967/hf/1200 },  
    publisher    = { Hugging Face } }

License

https://github.com/PhilipMay/stsb-multi-mt/blob/main/LICENSE