arbazsiddiqui/opensearch-neural-sparse-doc-v2-mini-GGUF

opensearch-neural-sparse-doc-v2-mini · GGUF

GGUF conversion of opensearch-project/opensearch-neural-sparse-encoding-doc-v2-mini (23M-parameter inference-free sparse retrieval encoder), packaged for sqlite-sparse: semantic search in a single SQLite file with no model at query time.

llama.cpp's BERT converter drops the masked-LM head that sparse encoders need, so this repo ships it as a sidecar: the .sprs file carries the MLM transform (dense + GELU + LayerNorm + tied decoder + bias), the model's query-weight lookup table (the "inference-free" query side), and the WordPiece vocabulary.

Files

Fidelity

Conversion is differentially validated against the original SentenceTransformers implementation:

• —encoder hidden states: cosine 0.999999 (f16), 0.999855 (Q8_0)
• —MLM head: max term-weight delta 6.9e-4 over 154 activated terms
• —document vectors through the compiled sqlite-sparse extension (Q8_0 encoder, this sidecar) vs the original torch pipeline: cosine 0.9992 to 0.9996 per document on SciFact abstracts
• —end to end through the extension at the card's 512-token setting: SciFact NDCG@10 0.6958 vs 0.699 on the original model card (Q8_0 encoder, u8 storage); 0.6929 at 256-token truncation

Usage

These files are the default model of the sqlite-sparse SQLite extension (documents are encoded once at index time; queries need only the weight table inside the .sprs). The pure-Python package uses the original HF checkpoint instead and treats this stack as its C-extension target. See sqlite-sparse for both paths.

Provenance

Weights are unmodified from the Apache-2.0 original by the OpenSearch project; this repo only changes container format (GGUF via llama.cpp's convert script; sidecar extracted with sqlite-sparse's converter). All retrieval-quality credit belongs to the original model.