Reports

Reviews of this box benchmark it for a week. We bought two, froze the workloads, and have been serving from them since: the software stack that works, what the numbers look like, how the two units compare, and the parts we still cannot vouch for.

August 21, 2026 · Funding: Self-funded internal research

The 256K-context configuration hunt on a single GB10: which vLLM attention and MoE backends survived SM_121, why NVFP4 decoded slower than FP8 in the mainline build we pinned, the patched setup that held 260K — and the measured numbers, dated to the versions named inside.

August 21, 2026 · Funding: Self-funded internal research

Every hardware guide for local AI is a spec sheet, a vendor reviewing itself, or an affiliate listicle recycling VRAM tiers. This one starts from the workload, uses only numbers we measured on hardware we own, and marks unmeasured lanes as gaps.

August 21, 2026 · Funding: Self-funded internal research

The story behind strizh-ru-retriever: layer-pruning RuModernBERT-small into a four-layer encoder, the distillation shortcut that produced a beautiful loss and dead retrieval, what the compression kept — and the leftover config value that silently truncated every input.

August 21, 2026 · Funding: Self-funded internal research

The story behind agmind-rag-splitter-ru: why a chunker that rewrites text is dangerous for RAG, how a LoRA fine-tune learned to return boundary indices as JSON, the training corpus published alongside it — and the honest gap between teacher agreement and retrieval quality.

August 21, 2026 · Funding: Self-funded internal research

The classic first failure on a Ryzen AI Max+ 395 box: the model does not fit, or the runtime reports a few gigabytes of VRAM on a 128 GB machine. The BIOS split is usually the wrong lever. Here is what our serving node actually reports, and the setting that matters.

August 21, 2026 · Funding: Self-funded internal research

Our two identical Strix Halo units live opposite lives: one carries the whole containerized service stack, the other serves a single model and stays otherwise empty. The split is not tidiness — the benchmark harness invalidates any run that finds a GPU neighbor on the host.

August 21, 2026 · Funding: Self-funded internal research

The advice circulating is written from datacenter GPU benchmarks and rarely survives contact with a single box. What each engine is actually for, which one we run where and why, and the defaults question that matters more than the choice itself.

August 21, 2026 · Funding: Self-funded internal research

Everyone asks whether a local model hallucinates over their documents; almost nobody measures it. Our workload has an unanswerable control built in: questions the document cannot answer, graded on whether the model admits it. Two model families, two ways to not know.

August 20, 2026 · Funding: Self-funded internal research

The most asked and least measured question in local AI, usually answered with invented round numbers. What speed actually feels like on a real box: the wait before the first useful word, the flow after it, and the failure that makes tokens per second irrelevant.

August 20, 2026 · Funding: Self-funded internal research

Most published local-LLM speeds cannot even be disagreed with: they carry no configuration. A practical protocol from a lab that publishes raw runs — what to pin, what to ask the running server, which tokens to count, and why failures belong in the denominator.

August 20, 2026 · Funding: Self-funded internal research

Strict-JSON output is the load-bearing wall of local automation, and its reliability is usually asserted, not measured. On one box: quantization did not move it, the backend did not move it, reasoning mode did not move it — switching the model family did.

August 20, 2026 · Funding: Self-funded internal research

Reasoning mode on a local model delays the first visible word from a blink to a wait and multiplies the cost of automation — and on every task with a hard right answer in our gates it bought nothing. The measured trade, and where it does not apply.

August 20, 2026 · Funding: Self-funded internal research

The most searched comparison in the local-AI mini-box class is answered everywhere by spec sheets and nowhere by one lab measuring both. We own both platforms. Here is what each one demonstrably did, which workload belongs on which box, and the comparison we refuse to fake.

August 20, 2026 · Funding: Self-funded internal research

The 128 GB unified-memory class is the current sweet spot for local LLM boxes, and almost everything written about it is spec-sheet arithmetic. Here is what one such box measurably does: which model class it serves, how fast, for how many people, and where 128 GB ends.

August 20, 2026 · Funding: Self-funded internal research

The exact serving recipe behind our published GB10 numbers: weights and image pinned, the head-first launch order that avoids a TCPStore hang, the MoE backend flag auto-selection skips, what to monitor when NVML goes silent — and the numbers to expect.

August 16, 2026 · Funding: Self-funded internal research

The standard GPU observability stack half-works on GB10 unified memory: dcgm-exporter does not run and NVML returns N/A on a large share of queries, so Grafana stays green and empty. The collector we actually serve vLLM with, and how to build it.

August 16, 2026 · Funding: Self-funded internal research

The highest-volume buyer question in this niche is answered almost entirely by spec-sheet arithmetic. Here is what a pair of GB10 nodes actually did in our lab — serving a 284B MoE, walking a context curve to a million tokens — and who should and should not buy one.

August 16, 2026 · Funding: Self-funded internal research

A document Q&A session on a Ryzen AI Max+ 395 box: what the first question over a 32k-token file costs, what the second one costs with the prompt cache on and off, and the one condition the cache needs before it saves you anything.

August 16, 2026 · Funding: Self-funded internal research

Same model, same llama.cpp build, same Strix Halo box, only the quantization changed: what Q8_0 costs in decode pace against Q4_K_M, what our quality gates saw, and which shortage should actually drive the choice.

August 16, 2026 · Funding: Self-funded internal research

A continuous three-hour serving pass on two identical Ryzen AI Max+ 395 units at concurrency four: how far the decode pace drifted between the first five minutes and the last, and what share of requests came back complete.

August 16, 2026 · Funding: Self-funded internal research

A reasoning model can consume its entire token budget thinking and return a successful response with zero user-visible text. We measured how often that happens on two model families — and why TTFT dashboards look perfect while it does.

August 14, 2026 · Funding: Self-funded internal research

Our long-context corpus was cut by a chars-per-token estimate that held for English and missed Russian by a third. The per-item measured counts are published — here is what they say about planning context windows for Cyrillic, and how we caught our own mislabel.

August 14, 2026 · Funding: Self-funded internal research

The same model on the same two GB10 nodes is reported anywhere from the high fifties to the low seventies of tokens per second. Every one of those figures can be correct — here is the configuration behind each, and which number to quote for which question.

August 13, 2026 · Funding: Self-funded internal research

A measured concurrency ladder on one Strix Halo unit: what the wait looks like at one, four and eight simultaneous chats, what stayed correct, and why 'tokens per second' cannot answer the sizing question at all.

August 12, 2026 · Funding: Self-funded internal research

Our fp8 DSpark bring-up on GB10, an independent reproduction of the community NVFP4 build, and the depth data nobody had published: the single-stream curve to a real million-token prompt — plus two negative results, stated as such.

August 5, 2026 · Funding: Self-funded internal research

Serving DeepSeek-V4-Flash-0731 across two DGX Spark nodes: the image ships a MoE kernel built for this model's MXFP4 experts on this silicon — and auto-selection skips it. Picking it by hand is worth double-digit percent of decode. Recipe and raw outputs are public.

August 3, 2026 · Funding: Self-funded internal research

Same Strix Halo unit, three swaps measured one at a time: a second model family reproduces the empty-answer failure mode; Q8_0 buys nothing measurable on these workloads; ROCm works on gfx1151 but decodes slower than Vulkan.

August 3, 2026 · Funding: Self-funded internal research

On a Strix Halo box the conventional TTFT figure looked excellent while most requests returned an empty answer with HTTP 200. Client-side measurement, three operating settings, evidence bundles included.

August 2, 2026 · Funding: Self-funded internal research

Bring-up of DSpark speculative decoding on GB10 (sm_121) over 200G RoCE, the 1M-token depth curve nobody had published, an independent reproduction of a community build, and two honest negative results.

July 31, 2026 · Funding: Self-funded internal research

What one 128 GB Strix Halo mini-PC did under 32 concurrent chat streams: recipes, an 8→10 concurrency cliff in every tested configuration, and where speculative decoding stops helping. Pre-methodology evidence.

July 31, 2026 · Funding: Self-funded internal research