The honest answer: “enough” is not one number, because the question hides three separate experiences — how long until the first useful word, how fast the text flows after it, and whether an answer arrives at all. Forums answer with round thresholds nobody measured. Here is what each experience actually is on one real box, with every figure carrying its configuration.
Why one tok/s number is the wrong lens
A single throughput figure averages over the two things a human actually feels: the pause before anything appears, and the pace once it does. Two setups with identical average tok/s can feel instant or broken depending on where the time sits. Worse, the most common speed number in the wild — decode tok/s from a synthetic benchmark — measures neither experience under a real prompt. So split the question the way perception splits it.
How long until the first useful word?
On everyday requests with reasoning off, this box shows its first answer token in 210msunit_replicated — below the threshold where a person registers waiting at all. The same model with reasoning on takes 20191msunit_replicated before the first visible word, while a naive dashboard watching the first token of anything would report 212msunit_replicated — because the thinking stream starts instantly and is not addressed to you. If you take one method away from this page: measure to the first answer token, or your metric will lie to you.
How fast should the answer flow?
Reading pace is the natural yardstick: text arriving faster than you read feels fluent, slower feels like dictation. This box decodes at 15.9ms/tokenrepeated on the everyday corpus — comfortably ahead of reading pace. Configuration moves this number more than hardware envy does: the heavier Q8_0 quant slows it to 18.7ms/tokenrepeated, and the ROCm backend to 18.7ms/tokenrepeated — the quant and backend choices are worth more than most upgrade plans.
Does the speed survive company and time?
Numbers taken alone, at minute one, flatter the box. Under four simultaneous chats the first answer token still arrives in 338msrepeated; at eight it is 865msrepeated — the full ladder is how to size a shared box. And over three continuous hours of serving, the decode pace drifted by 1.6%single_run — speed that does not fade with the workday.
When speed stops mattering
A model can be fast and still deliver nothing: under a tight token budget with reasoning on, this box returned 58.3% of requestsunit_replicated of requests as HTTP 200 with an empty answer — the budget went to thinking, none of it to you. A different model family in its default mode showed 8.3% of requestsrepeated on the same corpus. No tokens-per-second figure captures this, which is exactly why we measure it separately — and why a speed conversation that ignores it is incomplete.
So what is “enough”?
For solo chat on this class of hardware: the box above is past the point where speed is the bottleneck — first word under perception threshold, flow ahead of reading pace. For a shared box, read the concurrency ladder against the wait your users will tolerate. For automation, end-to-end time per completed, parseable answer is the metric — measured here. And for any setup with reasoning on, budget for the failure mode before the throughput.
If a page offers you one universal tok/s threshold instead of these measurements, it is quoting a mood. The method for measuring your own box is written up here; every number above has a permanent page with raw runs in the registry.