voidful/dpr-ctx_encoder-bert-base-multilingual

1---2language: multilingual3datasets:4- NQ5- Trivia6- SQuAD7- MLQA8- DRCD9---10 11# dpr-ctx_encoder-bert-base-multilingual12 13## Description14 15Multilingual DPR Model base on bert-base-multilingual-cased. 16[DPR model](https://arxiv.org/abs/2004.04906)17[DPR repo](https://github.com/facebookresearch/DPR)18 19## Data201. [NQ](https://github.com/facebookresearch/DPR/blob/master/data/download_data.py)212. [Trivia](https://github.com/facebookresearch/DPR/blob/master/data/download_data.py)223. [SQuAD](https://github.com/facebookresearch/DPR/blob/master/data/download_data.py)234. [DRCD*](https://github.com/DRCKnowledgeTeam/DRCD)245. [MLQA*](https://github.com/facebookresearch/MLQA)25 26`question pairs for train`： 644,217  27`question pairs for dev`： 73,71028 29*DRCD and MLQA are converted using script from haystack [squad_to_dpr.py](https://github.com/deepset-ai/haystack/blob/master/haystack/retriever/squad_to_dpr.py)30 31## Training Script32I use the script from [haystack](https://colab.research.google.com/github/deepset-ai/haystack/blob/master/tutorials/Tutorial9_DPR_training.ipynb)33 34## Usage35 36```python37from transformers import DPRContextEncoder, DPRContextEncoderTokenizer38tokenizer = DPRContextEncoderTokenizer.from_pretrained('voidful/dpr-ctx_encoder-bert-base-multilingual')39model = DPRContextEncoder.from_pretrained('voidful/dpr-ctx_encoder-bert-base-multilingual')40input_ids = tokenizer("Hello, is my dog cute ?", return_tensors='pt')["input_ids"]41embeddings = model(input_ids).pooler_output42```43 44Follow the tutorial from `haystack`:45[Better Retrievers via "Dense Passage Retrieval"](https://colab.research.google.com/github/deepset-ai/haystack/blob/master/tutorials/Tutorial6_Better_Retrieval_via_DPR.ipynb)46```47from haystack.retriever.dense import DensePassageRetriever48retriever = DensePassageRetriever(document_store=document_store,49                                  query_embedding_model="voidful/dpr-question_encoder-bert-base-multilingual",50                                  passage_embedding_model="voidful/dpr-ctx_encoder-bert-base-multilingual",51                                  max_seq_len_query=64,52                                  max_seq_len_passage=256,53                                  batch_size=16,54                                  use_gpu=True,55                                  embed_title=True,56                                  use_fast_tokenizers=True)57```58