datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
bookcorpus_filtered_len_17_simcsewikitext-103-raw-v1_sents_min_len10_max_len30_princeton-nlp_sup-simcse-roberta-largenli-for-simcse
Dataset Card for NLI for SimCSE
This is a reformatting of the NLI for SimCSE Dataset used to train the BGE-M3 model. See the full BGE-M3 dataset in Shitao/bge-m3-data.
Despite being labeled as Natural Language Inference (NLI), this dataset can be used for training/finetuning an embedding model for semantic textual similarity.
Dataset Subsets
triplet subset
Columns: "anchor", "positive", "negative"
Column types: str, str, str
Examples:{
'anchor': 'One… See the full description on the dataset page: https://huggingface.co/datasets/sentence-transformers/nli-for-simcse.cc12m_princeton-nlp_sup-simcse-roberta-largekor_nli_simcse
Korean Natural Language Inference (KorNLI) for SimCSE Dataset
For a better dataset description, please visit this GitHub repository prepared by the authors of the article: LINK
This dataset was prepared by converting KorNLI dataset. I took every unique premise of the dataset and searched for its entailment (positive example) and contradiction (negative example).
These changes have been made in order to apply SimCSE method.
I additionaly share the code, which I used to convert the… See the full description on the dataset page: https://huggingface.co/datasets/dkoterwa/kor_nli_simcse.jawiki-paragraphs-unsup-simcse-bert-base-japanese-v3
Dataset Card for "jawiki-paragraphs-unsup-simcse-bert-base-japanese-v3"
More Information needed
wiki1m-for-simcse
Dataset Card for Wiki1m for SimCSE
This is a reupload of the wiki1m_for_simcse.txt file from princeton-nlp/datasets-for-simcse, which can no longer be downloaded with recent datasets versions.
Columns: "text"
Column types: str
Examples:{'text': 'YMCA in South Australia'}
Collection strategy: Downloading the princeton-nlp/datasets-for-simcse dataset with datasets==2.21.0 and reuploading it to make the format compatible with datasets.
Deduplicated: No
flickr30k_CLIP_ViT-B-32_subset_pairs_SimCSE_similarity_copybookcorpus_filtered_len_17_simcse_retrieval_top32__source_tranch_10__target_tranch_10__from_120bookcorpus_filtered_len_17_simcse_retrieval_top32__source_tranch_19__target_tranch_26__from_120flickr30k_CLIP_ViT-B-32_subset_pairs_SimCSE_similaritymscoco_2014_captions_princeton-nlp_sup-simcse-roberta-largebookcorpus_filtered_len_17_simcse_retrieval_top32__source_tranch_14__target_tranch_9__from_120multipit_crowd_all_count_simcse_retrieval_pairsflickr30k_captions_simCSEbookcorpus_filtered_len_17_simcse_retrieval_top32__source_tranch_13__target_tranch_19__from_120bookcorpus_filtered_len_17_simcse_retrieval_top32__source_tranch_13__target_tranch_16__from_120bookcorpus_filtered_len_17_simcse_retrieval_top32__source_tranch_13__target_tranch_9__from_120klue-nli-simcse
KLUENLI for SimCSE Dataset
For a better dataset description, please visit: LINK
This dataset was prepared by converting KLUENLI dataset to use it for contrastive training (SimCSE). The code used to prepare the data is given below:
import pandas as pd
from datasets import load_dataset, concatenate_datasets, Dataset
from torch.utils.data import random_split
class PrepTriplets:
@staticmethod
def make_dataset():
train_dataset = load_dataset("klue", "nli"… See the full description on the dataset page: https://huggingface.co/datasets/phnyxlab/klue-nli-simcse.bookcorpus_filtered_len_17_simcse_retrieval_top32__source_tranch_19__target_tranch_27__from_120simcsemultipit_crowd_all_count_SimCSEmultipit_auto_SimCSEbookcorpus_filtered_len_17_simcse_retrieval_top32__source_tranch_19__target_tranch_28__from_120nli_for_simcsebookcorpus_filtered_len_17_simcse_retrieval_top32__source_tranch_10__target_tranch_28__from_120bookcorpus_filtered_len_17_simcse_retrieval_top32__source_tranch_10__target_tranch_2__from_120bookcorpus_filtered_len_17_simcse_retrieval_top32__source_tranch_11__target_tranch_3__from_120flickr30k_princeton-nlp_sup-simcse-roberta-largebookcorpus_filtered_len_17_simcse_retrieval_top32__source_tranch_12__target_tranch_9__from_120