# How many tokens per second is enough for a local LLM?

> The most asked and least measured question in local AI, usually answered with invented round numbers. What speed actually feels like on a real box: the wait before the first useful word, the flow after it, and the failure that makes tokens per second irrelevant.

- Published: 2026-08-20
- Evidence level: lab_repeated
- Funding: Self-funded internal research
- Canonical: https://agmind.ai/reports/how-many-tokens-per-second-is-enough/

The honest answer: "enough" is not one number, because the question hides
three separate experiences — how long until the first useful word, how fast
the text flows after it, and whether an answer arrives at all. Forums
answer with round thresholds nobody measured. Here is what each experience
actually is on one real box, with every figure carrying its configuration.

## Why one tok/s number is the wrong lens

A single throughput figure averages over the two things a human actually
feels: the pause before anything appears, and the pace once it does. Two
setups with identical average tok/s can feel instant or broken depending on
where the time sits. Worse, the most common speed number in the wild —
decode tok/s from a synthetic benchmark — measures neither experience under
a real prompt. So split the question the way perception splits it.

## How long until the first useful word?

On everyday requests with reasoning off, this box shows its first answer
token in **210 ms** ([strix.qwen36.interactive2.c1.ttfa-nothink](https://agmind.ai/claims/strix.qwen36.interactive2.c1.ttfa-nothink/)) —
below the threshold where a person registers waiting at all. The same
model with reasoning on takes
**20191 ms** ([strix.qwen36.interactive2.c1.ttfa-thinking](https://agmind.ai/claims/strix.qwen36.interactive2.c1.ttfa-thinking/)) before the
first visible word, while a naive dashboard watching the first token of
*anything* would report
**212 ms** ([strix.qwen36.interactive2.c1.ttft-any-token](https://agmind.ai/claims/strix.qwen36.interactive2.c1.ttft-any-token/)) — because
the thinking stream starts instantly and is not addressed to you. If you
take one method away from this page: measure to the first *answer* token,
or [your metric will lie to you](https://agmind.ai/reports/ttft-thinking-model-strix-halo/).

## How fast should the answer flow?

Reading pace is the natural yardstick: text arriving faster than you read
feels fluent, slower feels like dictation. This box decodes at
**15.9 ms/token** ([strix.qwen36q4.vulkan.c1.itl-nothink](https://agmind.ai/claims/strix.qwen36q4.vulkan.c1.itl-nothink/)) on the everyday
corpus — comfortably ahead of reading pace. Configuration moves this
number more than hardware envy does: the heavier Q8_0 quant slows it to
**18.7 ms/token** ([strix.qwen36q8.vulkan.c1.itl-nothink](https://agmind.ai/claims/strix.qwen36q8.vulkan.c1.itl-nothink/)), and the ROCm
backend to **18.7 ms/token** ([strix.qwen36q4.rocm.c1.itl-nothink](https://agmind.ai/claims/strix.qwen36q4.rocm.c1.itl-nothink/)) — the
[quant](https://agmind.ai/compare/q4km-vs-q80-qwen36-strix-halo/) and
[backend](https://agmind.ai/compare/llamacpp-vulkan-vs-rocm-strix-halo/) choices are worth
more than most upgrade plans.

## Does the speed survive company and time?

Numbers taken alone, at minute one, flatter the box. Under four
simultaneous chats the first answer token still arrives in
**338 ms** ([strix.qwen36.interactive2.c4.ttfa-nothink](https://agmind.ai/claims/strix.qwen36.interactive2.c4.ttfa-nothink/)); at eight it
is **865 ms** ([strix.qwen36.interactive2.c8.ttfa-nothink](https://agmind.ai/claims/strix.qwen36.interactive2.c8.ttfa-nothink/)) — the
[full ladder](https://agmind.ai/reports/concurrency-capacity-strix-halo/) is how to size a
shared box. And over three continuous hours of serving, the decode pace
drifted by
**1.6 %** ([strix.qwen36.endurance.c4.itl-drift-180m](https://agmind.ai/claims/strix.qwen36.endurance.c4.itl-drift-180m/)) — speed that
[does not fade with the workday](https://agmind.ai/reports/strix-halo-sustained-load-3-hours/).

## When speed stops mattering

A model can be fast and still deliver nothing: under a tight token budget
with reasoning on, this box returned
**58.3 % of requests** ([strix.qwen36.interactive2.c1.answerless-1k](https://agmind.ai/claims/strix.qwen36.interactive2.c1.answerless-1k/)) of requests
as HTTP 200 with an empty answer — the budget went to thinking, none of it
to you. A different model family in its default mode showed
**8.3 % of requests** ([strix.gemma4.interactive2.c1.answerless-default](https://agmind.ai/claims/strix.gemma4.interactive2.c1.answerless-default/)) on the
same corpus. No tokens-per-second figure captures this, which is exactly
why [we measure it separately](https://agmind.ai/reports/answerless-http-200/) — and why a
speed conversation that ignores it is incomplete.

## So what is "enough"?

For solo chat on this class of hardware: the box above is past the point
where speed is the bottleneck — first word under perception threshold,
flow ahead of reading pace. For a shared box, read the concurrency ladder
against the wait your users will tolerate. For automation, end-to-end time
per completed, *parseable* answer is the metric —
[measured here](https://agmind.ai/reports/local-llm-json-reliability/). And for any setup
with reasoning on, budget for the failure mode before the throughput.

If a page offers you one universal tok/s threshold instead of these
measurements, it is quoting a mood. The method for measuring your own box
is [written up here](https://agmind.ai/reports/how-to-benchmark-local-llm/); every number
above has a permanent page with raw runs in [the registry](https://agmind.ai/claims/).

---

Machine-readable claim registry: https://agmind.ai/claims.json · llms.txt: https://agmind.ai/llms.txt