# A Russian retrieval embedder in 24M parameters — and the one number that nearly ruined it

> The story behind strizh-ru-retriever: layer-pruning RuModernBERT-small into a four-layer encoder, the distillation shortcut that produced a beautiful loss and dead retrieval, what the compression kept — and the leftover config value that silently truncated every input.

- Published: 2026-08-21
- Evidence level: lab_single_run
- Funding: Self-funded internal research
- Canonical: https://agmind.ai/reports/russian-embedder-24m-params/

[strizh-ru-retriever](https://huggingface.co/AGmind/strizh-ru-retriever)
is the lab's compact Russian sentence encoder: 24.4M parameters, four layers, one 384-dimensional embedding per text ,
no query/document prefixes, published under Apache-2.0 with
[GGUF weights](https://huggingface.co/AGmind/strizh-ru-retriever-GGUF)
alongside. The full story of how it was made — including the mistake this
page is named after — is a Russian long-read on
[Habr](https://habr.com/ru/articles/1064138/), and that article is the
primary source for every figure quoted here. This is the English companion:
the same story, condensed, with the numbers marked as quotes from the
article rather than registry claims.

## Why shrink an embedder that already fits?

Not for memory. The base model is small enough to live anywhere; the thing
that ran out was compute. In a self-hosted RAG box the embedder shares
silicon with the generator, and every query it embeds — and every corpus it
reindexes — is stolen from the LLM's decode budget. As the article puts it,
the memory was not running out; GPU time was. The design target was a
first-stage retriever cheap enough to sit next to a serving LLM without
being felt, and simple enough that wrappers cannot misuse it — which is why
it takes bare text, with none of the query/document prefixes that every
integration forgets to add.

## What failed before anything worked

The base was `deepvk/RuModernBERT-small`: 12 transformer layers, about 35M parameters .
The obvious shortcut — regress the small model's embeddings onto a large
teacher's with MSE — produced the article's best cautionary tale. The loss
settled around 0.00062, and retrieval was dead on arrival: Recall@10 of 0.029 .
A proxy objective converged perfectly to something that was not the task.
The recipe that did work is contrastive end to end: train the 12-layer
donor as a retriever first, keep layers [0, 5, 9, 11] as a warm start for the student ,
then re-train the four-layer model on a 220-thousand-pair mixed Russian/English set
with hard negatives mined by a larger retriever.

## What did the compression keep?

The article's quality benchmark is MIRACL-ru dev, on a filtered subset of 758 of 1000 queries ;
Recall@10 as published there :

| Model | Recall@10 |
| --- | --- |
| bge-m3 | 0.831 |
| multilingual-e5-small | 0.829 |
| USER2-small | 0.819 |
| **strizh-ru-retriever** | **0.751** |
| rubert-tiny2 | 0.633 |

On the unfiltered set it reads 0.800 . So no, the small model does
not beat the large ones, and the article does not pretend it does: it gives
up a measured slice of recall against bge-m3 and buys back an order of
magnitude elsewhere. On a Strix Halo box under llama.cpp/Vulkan at concurrency 16 it embedded about four thousand short queries per second — 3,614 to 4,450 across runs — versus about 448 for bge-m3 and 1,837 for multilingual-e5-small .
Run online next to a serving 35B-class MoE at 200 embedding QPS, it cost the LLM about 2% of generation throughput where bge-m3 cost 7% ;
under a full reindex the co-resident LLM held 14.0 transactions per minute next to STRIZH against 6.0 next to bge-m3 .
The GGUF files are 52 MB at F16, 29 MB at Q8_0, 26 MB at Q4_0 .
That is the trade, stated with its price tag.

## The one wrong number

The near-miss is the part most worth retelling. The released
`tokenizer_config.json` carried `model_max_length: 256`
— a leftover from contrastive training, which used 256-token sequences on an architecture that accepts 8,192 .
Every standard wrapper reads that field and silently truncates. Nothing
crashes, nothing warns; the model simply never sees most of a long passage.
It surfaced during dogfooding on the lab's internal RAG corpus — 163 generated questions over its own documents
— where the same checkpoint scored Recall@10 of 0.589 with the bug fixed and 0.448 with it present: a drop of 14.1 percentage points ,
down into rubert-tiny2 territory. One digit in a config file nearly
cancelled the entire training effort, and no benchmark run on short
queries would ever have caught it.

That is this lab's founding observation wearing a different hat: a quality
number detached from its configuration is
[a config in disguise](https://agmind.ai/essays/benchmark-number-config-in-disguise/). The
checkpoint had one Recall@10; the checkpoint-plus-shipped-config had
another, and only the second one is real for users. The fix — and a release
checklist item that now diffs effective tokenizer settings against the
architecture — shipped before the public release did.

## Where everything lives, and what this page is not

The primary source is the
[Habr long-read](https://habr.com/ru/articles/1064138/) (Russian), with the
full training iterations, evaluation methodology and caveats. The model and
its GGUF builds are on Hugging Face, and the artifact's identity is on
[the artifacts page](https://agmind.ai/artifacts/). What this page is not: none of the
numbers above are lab registry claims. They are quotes from the article —
single-lab work, evaluated by its own author, with speed figures from one
box — and they keep the article's own caveats. What our registry-grade
protocol would demand of them is
[a separate report](https://agmind.ai/reports/how-to-benchmark-local-llm/).

---

Machine-readable claim registry: https://agmind.ai/claims.json · llms.txt: https://agmind.ai/llms.txt