uaritm/multilingual_en_uk_pl_ru

{MODEL_NAME}

This is a sentence-transformers model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.

<!--- Describe your model here -->

Usage (Sentence-Transformers)

The model is used on the resource of multilingual analysis of patient complaints to determine the specialty of the doctor that is needed in this case: Virtual General Practice

You can test the quality and speed of the model

This model is an updated version of the model: uaritm/multilingual_en_ru_uk

Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:

pip install -U sentence-transformers

Then you can use the model like this:

from sentence_transformers import SentenceTransformer sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('{MODEL_NAME}') embeddings = model.encode(sentences) print(embeddings)

## Usage (HuggingFace Transformers)
Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.

from transformers import AutoTokenizer, AutoModel import torch

#Mean Pooling - Take attention mask into account for correct averaging def meanpooling(modeloutput, attentionmask): tokenembeddings = modeloutput[0] #First element of modeloutput contains all token embeddings inputmaskexpanded = attentionmask.unsqueeze(-1).expand(tokenembeddings.size()).float() return torch.sum(tokenembeddings * inputmaskexpanded, 1) / torch.clamp(inputmask_expanded.sum(1), min=1e-9)

Sentences we want sentence embeddings for

sentences = ['This is an example sentence', 'Each sentence is converted']

Load model from HuggingFace Hub

tokenizer = AutoTokenizer.frompretrained('{MODELNAME}') model = AutoModel.frompretrained('{MODELNAME}')

Tokenize sentences

encodedinput = tokenizer(sentences, padding=True, truncation=True, returntensors='pt')

Compute token embeddings

with torch.nograd(): modeloutput = model(**encoded_input)

Perform pooling. In this case, mean pooling.

sentenceembeddings = meanpooling(modeloutput, encodedinput['attention_mask'])

print("Sentence embeddings:") print(sentence_embeddings)

## Evaluation Results

<!--- Describe how your model was evaluated -->

For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name={MODEL_NAME})


## Training
The model was trained with the parameters:

**DataLoader**:

`torch.utils.data.dataloader.DataLoader` of length 50184 with parameters:

{'batchsize': 32, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batchsampler': 'torch.utils.data.sampler.BatchSampler'}

**Loss**:

`sentence_transformers.losses.MSELoss.MSELoss` 

Parameters of the fit()-Method:

{ "epochs": 4, "evaluationsteps": 1000, "evaluator": "sentencetransformers.evaluation.SequentialEvaluator.SequentialEvaluator", "maxgradnorm": 1, "optimizerclass": "<class 'torch.optim.adamw.AdamW'>", "optimizerparams": { "eps": 1e-06, "lr": 2e-05 }, "scheduler": "WarmupLinear", "stepsperepoch": null, "warmupsteps": 10000, "weightdecay": 0.01 }

## Full Model Architecture

SentenceTransformer( (0): Transformer({'maxseqlength': 128, 'dolowercase': False}) with Transformer model: XLMRobertaModel (1): Pooling({'wordembeddingdimension': 768, 'poolingmodeclstoken': False, 'poolingmodemeantokens': True, 'poolingmodemaxtokens': False, 'poolingmodemeansqrtlentokens': False}) )

## Citing & Authors

@misc{UARITM, title={sentence-transformers: Semantic similarity of medical texts ukr, kor, eng}, author={Vitaliy Ostashko}, year={2023}, url={https://ai.esemi.org}, }

<!--- Describe where people can find more information -->