Qdrant/bm25

1---2license: apache-2.03language:4- en5- ar6- nl7- da8- fi9- fr10- de11- el12- hu13- it14- 'no'15- pt16- ro17- ru18- es19- sv20- ta21- tr22pipeline_tag: sentence-similarity23---24Repository with files to perform BM25 searches with [FastEmbed](https://github.com/qdrant/fastembed).25 26[BM25 (Best Matching 25)](https://en.wikipedia.org/wiki/Okapi_BM25) is a ranking function used by search engines to estimate the relevance of documents to a given search query.27 28### Usage29 30> Note:31This model is supposed to be used with Qdrant. Vectors have to be configured with [Modifier.IDF](https://qdrant.tech/documentation/concepts/indexing/?q=modifier#idf-modifier).32 33Here's an example of BM25 with [FastEmbed](https://github.com/qdrant/fastembed).34 35```py36from fastembed import SparseTextEmbedding37 38documents = [39    "You should stay, study and sprint.",40    "History can only prepare us to be surprised yet again.",41]42 43model = SparseTextEmbedding(model_name="Qdrant/bm25")44embeddings = list(model.embed(documents))45 46# [47#     SparseEmbedding(48#         values=array([1.67419738, 1.67419738, 1.67419738, 1.67419738]),49#         indices=array([171321964, 1881538586, 150760872, 1932363795])),50#     SparseEmbedding(values=array(51#         [1.66973021, 1.66973021, 1.66973021, 1.66973021, 1.66973021]),52#                     indices=array([53#                         578407224, 1849833631, 1008800696, 2090661150,54#                         111739301955#                     ]))56# ]57```58 59 60 61```