Datasetpublic

barbaroo/STS

This is a synthetic Faroese Semantic Textual Similarity dataset. Labels range from 0 (no similarity) to 5 (the two sentences are completely equivalent). The dataset was generated by: Translating sentences from the Basic Faroese Language Resource Kit (BLARK) corpus to English by leveraging a Nordic LLM, GPT-Sw3. Sentences were compared to each other in terms of semantic similarity by Sentence BERT (SBERT, ) Pairs of sentences were then sampled uniformly in terms of similarity score, to compile… See the full description on the dataset page: https://huggingface.co/datasets/barbaroo/STS.

sourceHugging Facecc-by-4.0updated 2y agoView on Hugging Face

0likes10downloads

Dataset Card

This is a synthetic Faroese Semantic Textual Similarity dataset. Labels range from 0 (no similarity) to 5 (the two sentences are completely equivalent). The dataset was generated by:

—Translating sentences from the Basic Faroese Language Resource Kit (BLARK) corpus to English by leveraging a Nordic LLM, GPT-Sw3.
—Sentences were compared to each other in terms of semantic similarity by Sentence BERT (SBERT, )
—Pairs of sentences were then sampled uniformly in terms of similarity score, to compile a balanced dataset. The dataset contains 200 sentences for each class (Similarity = 0,1,2,3,4,5).