sentence-transformers/embedding-training-data
Training Data for Text Embedding Models [!NOTE] This repository contains raw datasets, all of which have also been formatted for easy training in the Embedding Model Datasets collection. We recommend looking there first. This repository contains training files to train text embedding models, e.g. using sentence-transformers. Data Format All files are in a jsonl.gz format: Each line contains a JSON-object that represent one training example. The JSON objects can… See the full description on the dataset page: https://huggingface.co/datasets/sentence-transformers/embedding-training-data.
Training Data for Text Embedding Models
[!NOTE] This repository contains raw datasets, all of which have also been formatted for easy training in the Embedding Model Datasets collection. We recommend looking there first.
This repository contains training files to train text embedding models, e.g. using sentence-transformers.
Data Format
All files are in a jsonl.gz format: Each line contains a JSON-object that represent one training example.
The JSON objects can come in different formats:
- Pairs:
["text1", "text2"]- This is a positive pair that should be close in vector space. - Triplets:
["anchor", "positive", "negative"]- This is a triplet: Thepositivetext should be close to theanchor, while thenegativetext should be distant to theanchor. - Sets:
{"set": ["text1", "text2", ...]}A set of texts describing the same thing, e.g. different paraphrases of the same question, different captions for the same image. Any combination of the elements is considered as a positive pair. - Query-Pairs:
{"query": "text", "pos": ["text1", "text2", ...]}A query together with a set of positive texts. Can be formed to a pair["query", "positive"]by randomly selecting a text frompos. - Query-Triplets:
{"query": "text", "pos": ["text1", "text2", ...], "neg": ["text1", "text2", ...]}A query together with a set of positive texts and negative texts. Can be formed to a triplet["query", "positive", "negative"]by randomly selecting a text fromposandneg.
Available Datasets
Note: I'm currently in the process to upload the files. Please check again next week to get the full list of datasets
We measure the performance for each training dataset by training the nreimers/MiniLM-L6-H384-uncased model on it with MultipleNegativesRankingLoss, a batch size of 256, for 2000 training steps. The performance is then averaged across 14 sentence embedding benchmark datasets from diverse domains (Reddit, Twitter, News, Publications, E-Mails, ...).
Disclaimer: We only distribute these datasets in a specific format, but we do not vouch for their quality or fairness, or claim that you have license to use the dataset. It remains the user's responsibility to determine whether you as a user have permission to use the dataset under the dataset's license and to cite the right owner of the dataset. Please check the individual dataset webpages for the license agreements.
If you're a dataset owner and wish to update any part of it, or do not want your dataset to be included in this dataset collection, feel free to contact me.