Matjac5/MNLP_M3_document_encoder

1---2license: apache-2.03library_name: sentence-transformers4tags:5- sentence-transformers6- feature-extraction7- sentence-similarity8- transformers9pipeline_tag: sentence-similarity10---11 12# sentence-transformers/facebook-dpr-ctx_encoder-single-nq-base13 14This is a port of the [DPR Model](https://github.com/facebookresearch/DPR) to [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.15 16 17 18## Usage (Sentence-Transformers)19 20Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:21 22```23pip install -U sentence-transformers24```25 26Then you can use the model like this:27 28```python29from sentence_transformers import SentenceTransformer30sentences = ["This is an example sentence", "Each sentence is converted"]31 32model = SentenceTransformer('sentence-transformers/facebook-dpr-ctx_encoder-single-nq-base')33embeddings = model.encode(sentences)34print(embeddings)35```36 37 38 39## Usage (HuggingFace Transformers)40Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.41 42```python43from transformers import AutoTokenizer, AutoModel44import torch45 46 47def cls_pooling(model_output, attention_mask):48    return model_output[0][:,0]49 50 51# Sentences we want sentence embeddings for52sentences = ['This is an example sentence', 'Each sentence is converted']53 54# Load model from HuggingFace Hub55tokenizer = AutoTokenizer.from_pretrained('sentence-transformers/facebook-dpr-ctx_encoder-single-nq-base')56model = AutoModel.from_pretrained('sentence-transformers/facebook-dpr-ctx_encoder-single-nq-base')57 58# Tokenize sentences59encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')60 61# Compute token embeddings62with torch.no_grad():63    model_output = model(**encoded_input)64 65# Perform pooling. In this case, max pooling.66sentence_embeddings = cls_pooling(model_output, encoded_input['attention_mask'])67 68print("Sentence embeddings:")69print(sentence_embeddings)70```71 72 73 74## Full Model Architecture75```76SentenceTransformer(77  (0): Transformer({'max_seq_length': 509, 'do_lower_case': False}) with Transformer model: BertModel 78  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})79)80```81 82## Citing & Authors83 84Have a look at: [DPR Model](https://github.com/facebookresearch/DPR)