# What 128 GB of unified memory actually runs: local LLMs, measured

> The 128 GB unified-memory class is the current sweet spot for local LLM boxes, and almost everything written about it is spec-sheet arithmetic. Here is what one such box measurably does: which model class it serves, how fast, for how many people, and where 128 GB ends.

- Published: 2026-08-20
- Evidence level: lab_repeated
- Funding: Self-funded internal research
- Canonical: https://agmind.ai/reports/what-128gb-unified-memory-runs/

The short, measured answer: a 128 GB unified-memory box comfortably serves a
35B-class MoE model with a sub-second first answer token for a single
reader, holds a small group at usable latency, walks a 32k-token document
without drama, and does it all day without fading. What it does not buy is
the largest open model class — that took two such boxes joined by a fast
link, and the seams showed. The evidence for every clause follows.

Almost everything published about this hardware class answers a different
question: how much memory the spec sheet promises and what could fit in it
arithmetically. Fitting is not serving. Below is what one 128 GB box — a
Beelink GTR9 Pro with a Ryzen AI Max+ 395 and Radeon 8060S, llama.cpp
pinned by image digest — measurably did under frozen workloads, with every
number linking to its scope, limitations and raw runs.

## Which model class it serves, and how it feels

The workhorse of our registry is Qwen3.6-35B-A3B at Q4_K_M — a 35B-class
MoE whose weights occupy well under a third of the box's memory, leaving
room for context, cache and a second loaded model. On everyday requests it
answers in **210 ms** ([strix.qwen36.interactive2.c1.ttfa-nothink](https://agmind.ai/claims/strix.qwen36.interactive2.c1.ttfa-nothink/))
and decodes at
**15.9 ms/token** ([strix.qwen36q4.vulkan.c1.itl-nothink](https://agmind.ai/claims/strix.qwen36q4.vulkan.c1.itl-nothink/)) — a pace a single
reader experiences as fluent. The heavier Q8_0 quant also fits with room to
spare; what it does and does not buy is
[its own report](https://agmind.ai/reports/q8-0-vs-q4-k-m-strix-halo/).

A second model family, gemma-4-26B-A4B, is measured on the same box in
[the swap report](https://agmind.ai/reports/model-quant-backend-strix-halo/) — the point
being that on 128 GB, switching model families is a decision about quality
and behavior, not about whether the weights fit.

## Long documents

Memory capacity is only half of long-context work; the other half is the
prefill bill. The first question over a 32k-token document costs
**33965 ms** ([strix.qwen36.longctx.c1.ttft-32k-en](https://agmind.ai/claims/strix.qwen36.longctx.c1.ttft-32k-en/)) on this box, and
with the llama.cpp prompt cache doing its job the second question over the
same document costs
**860 ms** ([strix.qwen36.docsession.c1.ttft-q2-32k-cache](https://agmind.ai/claims/strix.qwen36.docsession.c1.ttft-q2-32k-cache/)). The
difference between a workflow that pays the fill once and one that pays it
per request is [its own story](https://agmind.ai/reports/llamacpp-prompt-cache-strix-halo/).

## More than one person

At four concurrent chats the median wait for the first answer token is
**338 ms** ([strix.qwen36.interactive2.c4.ttfa-nothink](https://agmind.ai/claims/strix.qwen36.interactive2.c4.ttfa-nothink/)), and at
eight it is **865 ms** ([strix.qwen36.interactive2.c8.ttfa-nothink](https://agmind.ai/claims/strix.qwen36.interactive2.c8.ttfa-nothink/)),
with completion holding at every step. The full ladder, and why a request
is not an employee, is in
[the capacity report](https://agmind.ai/reports/concurrency-capacity-strix-halo/).

## All day

Three continuous hours at concurrency four moved the decode pace by
**1.6 %** ([strix.qwen36.endurance.c4.itl-drift-180m](https://agmind.ai/claims/strix.qwen36.endurance.c4.itl-drift-180m/)) between the
first five minutes and the last — the sustained-load fade the mini-PC
skeptics predict [did not arrive](https://agmind.ai/reports/strix-halo-sustained-load-3-hours/)
on either of our two identical units.

## Where 128 GB ends

The largest open MoE class does not fit in one such box. Serving a
284B-parameter model took two 128 GB units in tensor parallel over a fast
link, a runtime flag the automatic mode skips, and real bring-up work — the
measured account is [the DGX Spark buyer answer](https://agmind.ai/reports/dgx-spark-worth-it/)
and [the pair's benchmark tables](https://github.com/botAGI/dgx-spark-llm-benchmarks).
One box serves the 35B class excellently; the frontier class is a
two-box project with seams.

## What this page does not claim

- **One vendor's box, one runtime family.** Other 128 GB unified-memory
  machines — Apple's, in particular — are declared lanes on
  [our testbed](https://agmind.ai/testbed/) with no published runs yet; we do not extrapolate
  to them.
- **No price advice.** Hardware prices move; the lab does not publish them.
- **Smaller tiers are unmeasured here.** What 8–24 GB of VRAM does to the
  same workloads is a different study, planned, not done.

Every number above has a permanent page with raw runs in
[the claim registry](https://agmind.ai/claims/); the full measured tables for this box live
in [the Strix Halo benchmark repository](https://github.com/botAGI/strix-halo-llm-benchmarks).

---

Machine-readable claim registry: https://agmind.ai/claims.json · llms.txt: https://agmind.ai/llms.txt