AGmind/strizh-ru-retriever

strizh-ru-retriever

A tiny (24.4M), no-prefix dense retriever for Russian. A 4-layer sentence encoder layer-pruned and re-trained from `deepvk/RuModernBERT-small`. It produces a single 384-dimensional embedding per text, needs no query/document prefixes, and its architecture accepts up to 8192 tokens (the contrastive training used 256-token sequences). Built as a fast first-stage retriever for self-hosted Russian RAG where the encoder shares one GPU with an LLM and a reranker (e.g. an AMD Strix Halo box) — a small, fast embedder returns GPU time to generation.

Honest positioning. strizh is a specialist, not a quality leader. It beats rubert-tiny2 on Russian retrieval (0.75 vs 0.63 on the clean subset below) and on Strix Halo / Vulkan throughput — but rubert-tiny2 is faster on CPU. Larger models beat strizh on quality: its sibling `deepvk/USER2-small` (+10M params, prefix-based), multilingual-e5-small and BAAI/bge-m3 are ahead on Russian, English and cross-lingual retrieval. Pick strizh when it is RU-first, when no prefixes must hold, and when first-stage speed on a shared GPU matters — behind hybrid search and a reranker. See When to use.

Specification

Usage

Drop-in — no prefixes, no instructions. Queries and documents are encoded the same way.

sentence-transformers

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("AGmind/strizh-ru-retriever")

query = "как включить mean-пулинг у эмбеддера"
docs = [
    "Для llama-server задайте --pooling mean, чтобы получить усреднённый эмбеддинг.",
    "Reranker сортирует кандидатов после первичного поиска.",
]

q = model.encode(query, normalize_embeddings=True)
d = model.encode(docs, normalize_embeddings=True)
print((d @ q))   # cosine similarities

Transformers

import torch, torch.nn.functional as F
from transformers import AutoTokenizer, AutoModel

tok = AutoTokenizer.from_pretrained("AGmind/strizh-ru-retriever")
model = AutoModel.from_pretrained("AGmind/strizh-ru-retriever")

def encode(texts):
    batch = tok(texts, padding=True, truncation=True, return_tensors="pt")
    with torch.no_grad():
        out = model(**batch).last_hidden_state
    mask = batch["attention_mask"].unsqueeze(-1).float()
    emb = (out * mask).sum(1) / mask.sum(1)      # mean pooling
    return F.normalize(emb, p=2, dim=1)

llama.cpp (GGUF, AMD / Vulkan / CPU)

llama-server -m strizh-ru-retriever.Q8_0.gguf \
  --embeddings --pooling mean -ngl 999 -c 8192 -b 4096 -ub 4096

GGUF weights: see `AGmind/strizh-ru-retriever-GGUF`. --pooling mean is required — the default cls collapses recall for this model.

Context caveat: -c is the total server context, divided across --parallel slots, and an embedding input must also fit into one ubatch (-ub). With -c 8192 -np 8 each slot gets only 1024 tokens and longer inputs return HTTP 400. To actually accept documents up to 8192 tokens with 8 slots use -c 65536 -b 8192 -ub 8192 -np 8 — the 4-layer KV cache is tiny, so this is cheap (verified: ~5000-token inputs embed fine).

Retrieval design

strizh implements one retrieval technology: dense, single-vector semantic search. This is deliberate. In a modern RAG stack the other retrieval modes are supplied by the surrounding components, so the embedder does not need to carry them:

Contrast bge-m3, which bundles all three modes into one 568M model. strizh takes the opposite bet: a tiny dense specialist that leaves sparse and re-ranking to the stack. Use it as the first-stage candidate generator in a hybrid + rerank pipeline.

Training

Derived from deepvk/RuModernBERT-small (which is itself Russian+English, Apache-2.0):

• —Warm-start layer pruning. The 12-layer donor is pruned to 4 layers ([0, 5, 9, 11]), keeping the shared 50 368-token embedding table, then re-trained with a contrastive objective — so the encoder is small in depth, not in vocabulary. (This is layer selection + warm-start + contrastive training, not classic online distillation.)
• —Contrastive objective. Multiple-Negatives Ranking Loss (in-batch negatives) plus hard negatives mined with bge-m3 from a mixed RU/EN candidate pool (the miner does not enforce that a negative is cross-lingual).
• —Data. Russian, English and mixed (Russian prose + code / English identifiers) pairs — MIRACL (Apache-2.0), opus-100, and ru_stackoverflow (CC BY-SA). Check each source's license for your use case; the base model is Apache-2.0.

Evaluation

RU retrieval uses our reproducible MIRACL-ru dev harness (first 1000 queries, dev passage corpus, 512-token cap, each model in its native pooling/prefix mode). strizh's donor training saw some MIRACL passages, so we report the passage-exposure-controlled subset — the 758 of 1000 queries whose gold passages never appeared as training positives — as the fair number. (On the full 1000-query set strizh scores 0.80; the filtered subset is also simply harder — every model scores lower on it — so the delta is not pure leakage, and the full-set score is best read as potentially optimistic due to that exposure.) EN = NanoBEIR mean nDCG@10 (via MTEB, prompt handling for prefix baselines relied on MTEB's model-metadata prompts — treat baseline EN numbers as approximate), not recall; cross-lingual = opus-100 translation-pair retrieval: ru↔en recall@10 (mean of both directions). Throughput = AMD Strix Halo, llama.cpp Vulkan, Q8_0, warmed, concurrency 16; across reproduced runs this point ranges 3614-4450 req/s, so read it as "about four thousand". Columns use different metrics — do not average across them. Bold = best in column.

Read honestly: strizh beats `rubert-tiny2` on Russian (0.75 vs 0.63) and is the fastest on the tested Vulkan stack — but it sits below USER2-small, mE5-small and bge-m3 on retrieval quality, and its English (0.34) and cross-lingual (0.76) axes are weak. A ru_stackoverflow "mixed" score exists too, but that eval overlaps the training data, so it is a training-diagnostic rather than a benchmark and is omitted here.

Real-world validation (dogfood RAG)

On a separate corpus from a different domain (the project's own repository — not part of strizh's retrieval training data; 1699 chunks of Russian docs + Python + config, 163 synthetic single-gold questions, bge-reranker-v2-m3 after retrieval), strizh reproduced the same picture end-to-end: it clearly beat rubert-tiny2 (recall@10 0.62 vs 0.53 after rerank) and stayed competitive with USER2-small on Russian strata, but cross-lingual retrieval stayed at 0.35 before rerank and 0.38 after it — if the right chunk is not in the candidate set, no reranker recovers it.

When to use

Use strizh when all of these hold:

• —the corpus is Russian-dominant;
• —you cannot (or do not want to) inject query/document prefixes — frozen RAG stacks, OpenAI-embeddings-shaped clients;
• —first-stage throughput/latency on a shared GPU is binding — the encoder shares one GPU with an LLM and a reranker, so a fast small embedder returns compute to generation;
• —it sits behind hybrid search and/or a reranker (as a first-stage retriever), not as the sole ranking signal.

Do not use strizh for:

• —cross-lingual retrieval (English query → Russian document, or vice versa) — 0.76;
• —English-heavy or mixed-language / code-symbol retrieval — 0.34 English;
• —CPU-only inference — rubert-tiny2 is faster there (357 vs 265 emb/s) and strizh has no ONNX path yet;
• —pure-vector stores with no lexical/rerank backstop when accuracy dominates over size.

If you can add prefixes, prefer USER2-small; if you have GPU/latency headroom, bge-m3 (also drop-in) is higher quality across the board.

GGUF quantizations

`AGmind/strizh-ru-retriever-GGUF`:

Just use Q8_0. The 50 368-token embedding table dominates the file and stays near 8-bit regardless, so Q50/Q40 save only ~2–3 MB over Q80 while losing quality — there is no reason to go below Q80 for this model. K-quants (Q4K / Q6K) are unavailable: hidden 384 / intermediate 576 are not multiples of the K-quant super-block (256), so llama-quantize silently falls back to legacy quants. Always serve with --pooling mean.

vLLM (NVIDIA)

The same checkpoint serves under vLLM (NVIDIA) as well as llama.cpp / Vulkan (AMD) and sentence-transformers (CPU/GPU). To serve under vLLM:

vllm serve AGmind/strizh-ru-retriever \
  --pooler-config '{"pooling_type":"MEAN"}' --max-model-len 8192

The repo ships the sentence-transformers layout (modules.json + pooling + normalize) and tokenizer_class = PreTrainedTokenizerFast so vLLM loads it as a MEAN-pooled embedding model. Pass truncate_prompt_tokens in requests if your inputs can exceed --max-model-len.

Limitations

• —Cross-lingual and English retrieval are weak (0.76 / 0.34). This is a RU-first model with residual English support, not an equal bilingual retriever.
• —Dense-only: it does not do lexical or multi-vector retrieval — pair it with hybrid search and a reranker.
• —Quality is below every larger Russian retriever; the value is speed on the target llama.cpp/Vulkan stack and drop-in (no-prefix) simplicity, not accuracy.

License

Model weights: Apache-2.0 (base deepvk/RuModernBERT-small is Apache-2.0). Note the training data mixes licenses — ru_stackoverflow is CC BY-SA, MIRACL is Apache-2.0 — so review data provenance for your own use case rather than assuming a single permissive license.

Citation

@misc{strizh_ru_retriever_2026,
  title  = {strizh-ru-retriever: a tiny drop-in dense retriever for Russian RAG},
  author = {AGmind},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/AGmind/strizh-ru-retriever}}
}