The short, measured answer: a 128 GB unified-memory box comfortably serves a 35B-class MoE model with a sub-second first answer token for a single reader, holds a small group at usable latency, walks a 32k-token document without drama, and does it all day without fading. What it does not buy is the largest open model class — that took two such boxes joined by a fast link, and the seams showed. The evidence for every clause follows.
Almost everything published about this hardware class answers a different question: how much memory the spec sheet promises and what could fit in it arithmetically. Fitting is not serving. Below is what one 128 GB box — a Beelink GTR9 Pro with a Ryzen AI Max+ 395 and Radeon 8060S, llama.cpp pinned by image digest — measurably did under frozen workloads, with every number linking to its scope, limitations and raw runs.
Which model class it serves, and how it feels
The workhorse of our registry is Qwen3.6-35B-A3B at Q4_K_M — a 35B-class MoE whose weights occupy well under a third of the box’s memory, leaving room for context, cache and a second loaded model. On everyday requests it answers in 210msunit_replicated and decodes at 15.9ms/tokenrepeated — a pace a single reader experiences as fluent. The heavier Q8_0 quant also fits with room to spare; what it does and does not buy is its own report.
A second model family, gemma-4-26B-A4B, is measured on the same box in the swap report — the point being that on 128 GB, switching model families is a decision about quality and behavior, not about whether the weights fit.
Long documents
Memory capacity is only half of long-context work; the other half is the prefill bill. The first question over a 32k-token document costs 33965msrepeated on this box, and with the llama.cpp prompt cache doing its job the second question over the same document costs 860msrepeated. The difference between a workflow that pays the fill once and one that pays it per request is its own story.
More than one person
At four concurrent chats the median wait for the first answer token is 338msrepeated, and at eight it is 865msrepeated, with completion holding at every step. The full ladder, and why a request is not an employee, is in the capacity report.
All day
Three continuous hours at concurrency four moved the decode pace by 1.6%single_run between the first five minutes and the last — the sustained-load fade the mini-PC skeptics predict did not arrive on either of our two identical units.
Where 128 GB ends
The largest open MoE class does not fit in one such box. Serving a 284B-parameter model took two 128 GB units in tensor parallel over a fast link, a runtime flag the automatic mode skips, and real bring-up work — the measured account is the DGX Spark buyer answer and the pair’s benchmark tables. One box serves the 35B class excellently; the frontier class is a two-box project with seams.
What this page does not claim
- One vendor’s box, one runtime family. Other 128 GB unified-memory machines — Apple’s, in particular — are declared lanes on our testbed with no published runs yet; we do not extrapolate to them.
- No price advice. Hardware prices move; the lab does not publish them.
- Smaller tiers are unmeasured here. What 8–24 GB of VRAM does to the same workloads is a different study, planned, not done.
Every number above has a permanent page with raw runs in the claim registry; the full measured tables for this box live in the Strix Halo benchmark repository.