# A 2k prompt vs a 32k prompt: what long context costs before the first token

Why is starting with a long document so much slower on a local llama.cpp server, and by how much?

The wait before the first token grows with document length: the model reads everything before saying anything, and the prefill bill scales with what it reads. Retrieval quality held at both depths on this workload. The escape hatch is the prompt cache — a repeated prefix pays the bill once, which is why the second question over the same document comes back almost immediately.

**What was held fixed.** Same unit, backend, artifact, quant and campaign; the only change is the document length in the prompt. Reasoning off, single stream.

| Metric | 2k-token document | 32k-token document |
| --- | --- | --- |
| Time to first token, median | 1907 ms ([lab_repeated](https://agmind.ai/claims/strix.qwen36.longctx.c1.ttft-2k-en/)) | 33965 ms ([lab_repeated](https://agmind.ai/claims/strix.qwen36.longctx.c1.ttft-32k-en/)) |

## Limits

Measured at 2k and 32k on one model and quant — no claim beyond 32k, and the curve between the two points is not characterized. Prefill-heavy workloads on other backends may scale differently.

## Related evidence

- https://agmind.ai/reports/llamacpp-prompt-cache-strix-halo/
- https://agmind.ai/workloads/long-context-v1/

Every number above is re-derived from sealed run bundles in CI: https://agmind.ai/claims.json (CC BY 4.0).
Source page: https://agmind.ai/compare/context-2k-vs-32k-qwen36-strix-halo/