demdecuong/stroke_simcse

1This is finetune version of [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://arxiv.org/abs/2104.08821)2, train unsupervised on 570K stroke sentences from : stroke books, quora medical, quora's stroke and human annotates.3 4### Extract sentence representation5```6from transformers import AutoTokenizer, AutoModel  7tokenizer = AutoTokenizer.from_pretrained("demdecuong/stroke_simcse")8model = AutoModel.from_pretrained("demdecuong/stroke_simcse")9 10text = "What are disease related to red stroke's causes?"11inputs = tokenizer(text, return_tensors='pt')12outputs = model(**inputs)[1]13```14### Build up embedding for database15 16```17database = [18    'What is the daily checklist for stroke returning home',19    'What are some tips for stroke adapt new life',20    'What  should I consider when using nursing-home care'21]22 23embedding = torch.zeros((len(database),768))24 25for i in range(len(database)):26  inputs = tokenizer(database[i], return_tensors="pt")27  outputs = model(**inputs)[1]28  embedding[i] = outputs29 30print(embedding.shape)31```32 33### Result34On our Poc testset , which contains pairs of matching question related to stroke from human-generated. 35 36| Model  | Top-1 Accuracy |37| ------------- | ------------- |38| SimCSE (supervised)  | 75.83  |39| SimCSE (ours)  | 76.66  |