# Q8_0 or Q4_K_M? What the bigger quant actually buys you, measured on one box

> Same model, same llama.cpp build, same Strix Halo box, only the quantization changed: what Q8_0 costs in decode pace against Q4_K_M, what our quality gates saw, and which shortage should actually drive the choice.

- Published: 2026-08-16
- Evidence level: lab_repeated
- Funding: Self-funded internal research
- Canonical: https://agmind.ai/reports/q8-0-vs-q4-k-m-strix-halo/

The short answer from a controlled cell: the bigger quant bought nothing our
quality gates could detect, and it decodes slower. On this box, for these
workloads, Q4_K_M wins on both counts. The long answer, with the numbers and
their limits, follows.

Every local LLM community relitigates this weekly. Someone downloads a model,
sees six quant files, asks which one, and receives a wall of confident
answers backed by perplexity tables from different hardware, different
models, or nothing. What the thread almost never contains is the same model
measured at two quants on the same machine with everything else frozen. That
is this page.

## The controlled cell

One Beelink GTR9 Pro, Ryzen AI Max+ 395 with 128 GB of unified memory.
llama.cpp server, Vulkan backend, build pinned by image digest. Qwen3.6-35B-A3B
from the same artifact family, once at Q4_K_M and once at Q8_0, each pinned
by hash. Same workloads, same gates, three repeated runs each, reasoning off.
The quantization is the only variable.

## Is Q8_0 slower than Q4_K_M?

Decode pace is where you feel a quant all day, so start there. Q4_K_M runs at
**15.9 ms/token** ([strix.qwen36q4.vulkan.c1.itl-nothink](https://agmind.ai/claims/strix.qwen36q4.vulkan.c1.itl-nothink/)). Q8_0 on the same
backend runs at **18.7 ms/token** ([strix.qwen36q8.vulkan.c1.itl-nothink](https://agmind.ai/claims/strix.qwen36q8.vulkan.c1.itl-nothink/)).

The bigger weights push more bytes through the same memory bus for every
token, and on a unified-memory box the bus is the ceiling. You pay it on
every token of every answer, forever. Both paces are usable for a single
reader; the gap is real and permanent.

An aside the backend curious will enjoy: Q8_0 on Vulkan lands at the same
pace as Q4_K_M on ROCm,
**18.7 ms/token** ([strix.qwen36q4.rocm.c1.itl-nothink](https://agmind.ai/claims/strix.qwen36q4.rocm.c1.itl-nothink/)) — picking the wrong
backend costs about as much as doubling your weights. That story continues in
[the backend comparison](https://agmind.ai/compare/llamacpp-vulkan-vs-rocm-strix-halo/).

## Does Q8_0 improve quality over Q4_K_M?

On our gates: nothing detectable. Strict-JSON automation, the workload where
degraded weights show up first as malformed output and wrong labels, passed
at **100.0 % of requests** ([strix.qwen36.structured.c1.task-success-nothink](https://agmind.ai/claims/strix.qwen36.structured.c1.task-success-nothink/)) on
Q4_K_M and at **100.0 % of requests** ([strix.qwen36q8.vulkan.c1.task-success](https://agmind.ai/claims/strix.qwen36q8.vulkan.c1.task-success/)) on
Q8_0. Identical. Format, language and repetition gates on the interactive
corpus: no difference either.

Read that claim as narrowly as it is written. Our gates check whether output
parses, matches ground truth on closed label sets, and stays in the right
language without loops. They are not a reasoning benchmark, not a knowledge
probe, and they say nothing about the subtle quality differences long-form
writing might surface. What they establish is that on tasks with a hard
right answer, this model at Q4_K_M gave Q8_0 nothing to win.

## Which quant should you use: Q8_0 or Q4_K_M?

Decide by what you are short of. Short of memory: Q4_K_M, and this is most
people. The Q8_0 artifact is substantially heavier on disk and in memory,
which on smaller boxes decides the question before quality gets a vote, and
even on 128 GB it can be the difference between fitting a second model
alongside or not.
Short of patience: Q4_K_M again, and that is the uncomfortable finding. The
default advice of "run the biggest quant that fits" spends your decode pace
on a quality difference our hard-answer gates could not detect. If your
workload is long-form prose where nuance might actually differ, run your own
corpus through both — [the harness is public](https://github.com/botAGI/agmind-bench).

## What this does not say

- **One model family.** MoE models with small active sets are widely held to
  quantize gracefully; a dense model may repay Q8_0 differently. We measured
  this model, and claim only it.
- **No perplexity, no accuracy suites.** Deliberately: those numbers travel
  badly across hardware and tokenizers. Task gates with ground truth are
  cruder and honester.
- **One box, one build.** The pace gap shape should hold on any
  bandwidth-limited unified-memory machine; exact figures are this box only.

The full matrix with both quants and both backends is on
[the comparison page](https://agmind.ai/compare/q4km-vs-q80-qwen36-strix-halo/), the deep
dive on quant-times-backend interaction in
[the quant and backend report](https://agmind.ai/reports/model-quant-backend-strix-halo/).
Every number above has a permanent page with scope and raw runs:
[the claim registry](https://agmind.ai/claims/).

---

Machine-readable claim registry: https://agmind.ai/claims.json · llms.txt: https://agmind.ai/llms.txt