FAISS
Datasets
All datasets matching “FAISS”faiss-smollm
FAISS-Based Novelty Detection for SmolLM and SmolLM2
This tutorial demonstrates how to the measure novelty of text queries with respect to the provided SmolLM and SmolLM2 pretraining corpora, with optional ColBERTv2 re-ranking for improved precision.
Overview
The pipeline consists of four main steps:
Generate Embeddings - Encode your queries using a sentence transformer
FAISS Search - Retrieve top-K most similar documents from the pretraining corpus
Combine… See the full description on the dataset page: https://huggingface.co/datasets/stai-tuebingen/faiss-smollm.pubmed-faiss-indexesfaiss_indexThe faiss index is built for fast retrieval using the embedding computed by ProTrek_650M_UniRef50.
Please download the faiss index and put it into the faiss_index directory, i.e. faiss_index/SwissProt/.... For more details, see our repo.
faiss_indicesfaiss-integration-testwikipedia_en_top_nopic_faiss