# Local LLM homelab hardware, measured: what to buy for what workload

> Every hardware guide for local AI is a spec sheet, a vendor reviewing itself, or an affiliate listicle recycling VRAM tiers. This one starts from the workload, uses only numbers we measured on hardware we own, and marks unmeasured lanes as gaps.

- Published: 2026-08-21
- Evidence level: lab_repeated
- Funding: Self-funded internal research
- Canonical: https://agmind.ai/reports/local-llm-homelab-hardware/

The short version: pick the workload first, then the box. A single-user
assistant, a team chat box and a million-token research rig are three
different purchases, and the number that decides each one is different. Below
is what our hardware measurably did for each workload — and, just as
importantly, the workloads where we have no data and refuse to invent it.

## Why hardware guides for local AI are so bad

Search any variant of "best hardware for local LLM" and the results share a
structure: a table of GPUs by VRAM, a paragraph of memory arithmetic, a
recommendation. Look for the measurement behind the recommendation and there
usually isn't one. Vendors rank for their own product categories — the top
results for mini PCs are frequently the mini-PC sellers reviewing themselves.
Affiliate roundups quote tokens-per-second figures that trace back, through
several hops, to two or three original sources, stripped of the model, quant,
context length and runtime that produced them. That stripping is the whole
problem: [a benchmark number is a config in
disguise](https://agmind.ai/essays/benchmark-number-config-in-disguise/), and a config-free
number cannot inform a purchase.

This page is narrower and, we think, more useful: what the two classes of
hardware we own actually did, under frozen workloads, with every figure
linked to its raw runs.

## Start from the workload, not the box

Four questions decide almost every local-AI hardware choice. Answer them
before looking at a single spec sheet.

**How many people?** One person's assistant and a five-person team are
different machines — not because of raw speed but because of how the wait
grows under concurrency.

**How long are the prompts?** A chat workload and a "paste a 50-page
document" workload stress different parts of the machine. The second is
dominated by prefill, which behaves nothing like generation.

**How big is the model, really?** Not the parameter count — the class. A
35B-class mixture-of-experts model and a 284B one live on opposite sides of
a boundary that no amount of unified memory in one box crosses.

**Is it on all day?** A box that is fast for ten minutes and a box that
holds its pace for eight hours are different products, and the difference
appears in no spec sheet.

## What a 128 GB unified-memory box does

Our workhorse class: a Ryzen AI Max+ 395 machine with 128 GB of unified
memory, llama.cpp pinned by image digest, serving a 35B-class MoE model.
Everything here is measured on frozen workloads with failures kept in the
denominator.

**Solo assistant.** First answer token in
**210 ms** ([strix.qwen36.interactive2.c1.ttfa-nothink](https://agmind.ai/claims/strix.qwen36.interactive2.c1.ttfa-nothink/)), decoding at
**15.9 ms/token** ([strix.qwen36q4.vulkan.c1.itl-nothink](https://agmind.ai/claims/strix.qwen36q4.vulkan.c1.itl-nothink/)) — under the
threshold where a person registers waiting, and ahead of reading pace. For
one user on everyday tasks, this class of box stopped being the bottleneck.

**A small team.** At four simultaneous chats the median wait for the first
word is **338 ms** ([strix.qwen36.interactive2.c4.ttfa-nothink](https://agmind.ai/claims/strix.qwen36.interactive2.c4.ttfa-nothink/)); at
eight it is **865 ms** ([strix.qwen36.interactive2.c8.ttfa-nothink](https://agmind.ai/claims/strix.qwen36.interactive2.c8.ttfa-nothink/)),
with completion holding at
**100.0 % of requests** ([strix.qwen36.interactive2.c8.completion-nothink](https://agmind.ai/claims/strix.qwen36.interactive2.c8.completion-nothink/)). The
[full ladder](https://agmind.ai/reports/concurrency-capacity-strix-halo/) is how to size this
honestly: pick the wait your users tolerate and read it backwards.

**Long documents.** The first question over a 32k-token document costs
**33965 ms** ([strix.qwen36.longctx.c1.ttft-32k-en](https://agmind.ai/claims/strix.qwen36.longctx.c1.ttft-32k-en/)) — that is the
prefill bill, and it is the number document-workflow buyers should look at
instead of decode speed. Stay in the session and the prompt cache brings the
next question back in
**860 ms** ([strix.qwen36.docsession.c1.ttft-q2-32k-cache](https://agmind.ai/claims/strix.qwen36.docsession.c1.ttft-q2-32k-cache/));
[the mechanics, and the one condition it
needs](https://agmind.ai/reports/llamacpp-prompt-cache-strix-halo/), matter more than
hardware here.

**All day.** Three continuous hours at concurrency four moved the decode
pace by **1.6 %** ([strix.qwen36.endurance.c4.itl-drift-180m](https://agmind.ai/claims/strix.qwen36.endurance.c4.itl-drift-180m/)), on
two commercially identical units. The
[sustained-load report](https://agmind.ai/reports/strix-halo-sustained-load-3-hours/) answers
"will a mini PC throttle if I leave it serving".

**Automation.** Strict-JSON tasks completed at
**100.0 % of requests** ([strix.qwen36.structured.c1.task-success-nothink](https://agmind.ai/claims/strix.qwen36.structured.c1.task-success-nothink/)) — and
[the reliability variable turned out to be the model, not the
settings](https://agmind.ai/reports/local-llm-json-reliability/).

What this memory class runs in general is
[its own page](https://agmind.ai/reports/what-128gb-unified-memory-runs/); the complete
measured tables live in
[the benchmark repository](https://github.com/botAGI/strix-halo-llm-benchmarks).

## What a pair of DGX Sparks does

The other class we own: two GB10 units joined over a 200G link, serving a
284B-parameter MoE through vLLM. This answers a specific question — running
the frontier open-weights class under your own roof — and it comes with
operational homework: a
[kernel-selection flag the automatic mode
skips](https://agmind.ai/reports/dsv4-flash-moe-backend-dgx-spark/),
[stock GPU monitoring that half-works](https://agmind.ai/reports/dgx-spark-gpu-monitoring/),
and a [context curve to a million
tokens](https://agmind.ai/reports/dspark-speculative-1m-dgx-spark/) whose first fill is an
event you schedule rather than a request you fire. The measured buyer
answer, including who should not buy one, is
[its own page](https://agmind.ai/reports/dgx-spark-worth-it/); the two-column comparison with
the Strix box is [here](https://agmind.ai/reports/strix-halo-vs-dgx-spark/).

## Which box for which workload

Reading our own measurements back as a decision:

- **One user, chat and code assistance** — the 128 GB unified class does
  this with room to spare. The choices that follow matter more than the box:
  [quantization](https://agmind.ai/reports/q8-0-vs-q4-k-m-strix-halo/) and
  [backend](https://agmind.ai/compare/llamacpp-vulkan-vs-rocm-strix-halo/) both moved our
  numbers more than most upgrade plans would.
- **A small team on chat-shaped work** — same class; size it from the
  concurrency ladder rather than a seat count.
- **Document-heavy work** — the same class again, but plan around prefill,
  not decode, and treat session reuse as a hardware-grade optimization.
- **The frontier open-weights class** — one box does not do it. A pair did,
  with seams, and we published both the recipe and the traps.
- **Latency-critical single-stream work on heavy prompts** — read the
  prefill figures above before buying anything; this is where local hardware
  disappoints people most often.

## What we have not measured

This is the section every other hardware guide omits, and it is why ours is
short where theirs are confident.

**Discrete-GPU builds.** Single and dual RTX-class cards, the used-3090
question, the 5090 question — these dominate the search results and we have
no primary data on them. Our [testbed](https://agmind.ai/testbed/) declares a CUDA lane; it
has no published runs, so we make no claims. When it runs, it will land with
raw runs like everything else.

**Apple Silicon.** Declared lane, zero published runs, zero claims.

**CPU-only and small-VRAM offload.** The "model doesn't fit" regime is the
most-asked and least-measured question below this hardware class. We have
not run it.

**Power and money.** We publish no prices — hardware prices move and a lab
that quotes them ages badly. Watts per token is a real gap we intend to
close once the metering methodology is settled; until then we have no
electricity figures either.

If a guide covers all of these confidently, check whether it measured any of
them.

## How to buy without trusting anyone, including us

Decide the workload, then demand of any number you read: what device and
firmware, what runtime build, what model artifact and quant, what prompt and
output length, what concurrency, which side of the wire, how many repeats.
[The protocol is written up here](https://agmind.ai/reports/how-to-benchmark-local-llm/) and
[the harness is public](https://github.com/botAGI/agmind-bench) — the frozen
workloads we run are the ones you can run on a box before or after buying it.

Common questions with one-line measured answers are indexed on
[the answers page](https://agmind.ai/answers/); every figure above has a permanent page with
its scope, limitations and raw evidence in [the claim registry](https://agmind.ai/claims/).

---

Machine-readable claim registry: https://agmind.ai/claims.json · llms.txt: https://agmind.ai/llms.txt