Controlled comparison
Prompt cache on vs off: the second question over the same document
Does the second question over the same pasted document pay the full prefill again on a local llama.cpp server?
What was held fixed
Same unit, backend, artifact; question pairs share a byte-identical document prefix. The only change is the server prompt-cache flag.
| Metric | Cache on | Cache off |
|---|---|---|
| TTFT, 2nd question over the same 32k document | 860msmedian over valid requests for this item · lab_repeated · evidence | 33728msmedian over valid requests for this item · lab_repeated · evidence |
| TTFT, 2nd question over the same 8k document (cache on) vs 1st question over a fresh 32k document | 660msmedian over valid requests for this item · lab_repeated · evidence | 33940msmedian over valid requests for this item · lab_repeated · evidence |
How to read this
With the cache on, the second question over a large document answers in under a second; with it off, the session pays the full half-minute prefill again. For paste-a-document workflows this flag is the difference between an interactive tool and an unusable one.
Limits
Byte-identical prefixes only — the cache buys nothing for a different or edited document. Single user, single stream; multi-user cache isolation was not measured.
Answering this question somewhere? Paste the table
Markdown, renders on GitHub / Reddit / forums. Attribution line included — the data is CC BY 4.0, take it.