# Multi-slot LLM inference on Strix Halo — legacy results

> What one 128 GB Strix Halo mini-PC did under 32 concurrent chat streams: recipes, an 8→10 concurrency cliff in every tested configuration, and where speculative decoding stops helping. Pre-methodology evidence.

- Published: 2026-07-31
- Evidence level: legacy
- Funding: Self-funded internal research
- Canonical: https://agmind.ai/reports/strix-halo-multislot-legacy/

> **Evidence level: `legacy`.** These measurements predate methodology v1: they
> were taken on this lab's own testbed with pinned image digests, salted
> uncached prompts and mostly 3-run medians — but without preregistration,
> without unit-to-unit replication, and on a software stack older than the
> current aligned one (kernel 6.17 vs today's 7.0). Numbers below are quoted
> from the public repository, not generated from the lab's claim registry.
> The flagship study re-measures this ground under methodology v1.

**Source (recipes, harnesses, raw per-run data):**
[github.com/botAGI/strix-halo-multislot](https://github.com/botAGI/strix-halo-multislot) (MIT)
· Narrative write-up (in Russian): [Habr article](https://habr.com/ru/articles/1060520/)

## What was tested

One Beelink GTR9 Pro (Ryzen AI Max+ 395, Radeon 8060S gfx1151, 128 GB unified
LPDDR5X) serving multi-user chat with llama.cpp `server-vulkan` at a pinned
image digest. Load generated over the network from a neighboring node; metrics
from server `timings`; aggregate throughput = Σ predicted tokens / makespan.
The repo publishes launch scripts, harnesses, full tables — including failures
and three of its own early claims that better measurements later overturned.

## Headline historical results (quoted from the repo)

| Scenario | Config | Repo-reported result |
|---|---|---|
| Multi-user chat, screening | Gemma 4 26B A4B, 32 clients | 236 tok/s aggregate, ~7 tok/s per request  |
| Endurance, 30 min | same | 226 tok/s average, 78°C, no throttling  |
| Multi-user Qwen | Qwen3.6-35B-A3B Q4_0, 32 clients | 178 tok/s aggregate (median of 3)  |
| Single stream + MTP draft | Qwen3.6 UD-Q4_K_M | ~90 tok/s per stream (median of 3)  |

## Findings worth knowing (testbed-scoped)

1. **A concurrency cliff right after 8 parallel requests appeared in all four
   tested configurations** — both model architectures, all three quants. The
   model determined the depth of the dip and the recovery shape. Root cause was
   not established (no kernel profiling); the practical mitigation measured for
   one model (capping active slots) would hurt the other.
2. **Speculative decoding (MTP) won single-stream and lost under load** at the
   measured points — a crossover that vendor materials, reporting large MTP
   gains on other stacks, would not predict for this one.
3. **Quantizations are not interchangeable under concurrency**, and an early
   single-run conclusion about the "best" quant was overturned by repeats —
   published in the repo as a refuted hypothesis.
4. **A four-parameter kernel tuning set bought a measurable generation gain**
   in a reboot A/B; per-parameter contributions were deliberately not claimed.
5. **llama.cpp's `--parallel` defaults to auto (4 slots in the tested builds)**
   — a server started "single-user" is already multi-slot. Comparing configs by
   flag lists nearly produced a false finding; `/props → total_slots` is the
   truth source.

## What this does not establish

No client-side TTFT (server-side `timings` only — queue wait excluded); no
quality evaluation of any configuration; no unit-to-unit replication; no
generalization beyond this exact box, image digest and kernel. The flagship
study (see the preregistration when published) covers exactly these gaps.

---

Machine-readable claim registry: https://agmind.ai/claims.json · llms.txt: https://agmind.ai/llms.txt