datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
NanoMTEB-Scandinavian
NanoMTEB-Scandinavian
This dataset is a Nano-style retrieval dataset for HAKARI-bench.
NanoMTEB-Scandinavian is a compact retrieval benchmark for Scandinavian-language MTEB-style task families. It includes Danish, Norwegian, and Swedish retrieval tasks spanning fact verification, question answering, news, encyclopedic content, FAQ retrieval, and social-media retrieval.
Usage
from datasets import load_dataset
dataset_id = "hakari-bench/NanoMTEB-Scandinavian"
split… See the full description on the dataset page: https://huggingface.co/datasets/hakari-bench/NanoMTEB-Scandinavian.scandinavian_faroesescandinavian-embedding-training-datascandinavian-100hscandinavian-educational-annotations
Scandinavian Educational Annotations
Created using a CommonCrawl dump (April 2024), and annotations with Gemini 1.5 Flash.
scandinavian-linguistic-annotations
Scandinavian Educational Annotations
Created using a CommonCrawl dump (April 2024), and annotations with Gemini 1.5 Flash.
europarl-scandinavian
Europarl Scandinavian Languages
The data originates from the Europarl parallel corpus, where English transcriptions of parliamentary discussions were aligned
with a number of other languages algorithmically.
In order to align Danish and Swedish corpora in the dataset, English entries were hashed with 128bit Murmurhash3,
and the Danish and Swedish transcriptions were joined on the obtained hash values.
Entries that had more than one pair in the other dataset were removed, this… See the full description on the dataset page: https://huggingface.co/datasets/kardosdrur/europarl-scandinavian.scandinavian-25h