# DeepSeek-V4-Flash DSpark on 2× DGX Spark: 1M context — legacy results

> Bring-up of DSpark speculative decoding on GB10 (sm_121) over 200G RoCE, the 1M-token depth curve nobody had published, an independent reproduction of a community build, and two honest negative results.

- Published: 2026-07-31
- Evidence level: legacy
- Funding: Self-funded internal research
- Canonical: https://agmind.ai/reports/dspark-dgx-spark-1m-legacy/

> **Evidence level: `legacy`.** Measured on this lab's own 2× DGX Spark cluster
> before methodology v1 — apples-to-apples within the repo
> (completion_tokens / wall_time, non-streaming), but without preregistration.
> Numbers are quoted from the public repository, not from the lab's claim
> registry.

**Source (patches, launchers, benchmark checkpoints, raw data):**
[github.com/botAGI/DeepSeek-V4-Flash-DSpark-GB10-2x-DGX-Spark-1m-fp4-fp8](https://github.com/botAGI/DeepSeek-V4-Flash-DSpark-GB10-2x-DGX-Spark-1m-fp4-fp8) (MIT)
· Narrative write-ups (in Russian): [the port and the one-line bug](https://habr.com/ru/articles/1055792/), [multi-user serving](https://habr.com/ru/articles/1050470/)

## What this work is

Two things, credited separately (full chain in the repo's CREDITS):

1. **Our own fp8 DSpark bring-up on GB10/sm_121** — DeepSeek's DSpark draft
   head and proposer wired into vLLM's spec-decode loop as patches (no fork of
   the base model), TP=2 across two DGX Spark units over 200G RoCEv2, at
   1M-token context. Includes the numerical bug-fixes that took the draft from
   "predicts noise" to "matches the reference".
2. **An independent reproduction and extended measurement of tonyd2wild's
   NVFP4-KV build** — explicitly not authored here; reproduced byte-identically
   and then measured beyond what had been published.

## Historical results (quoted from the repo)

- **fp8 DSpark (ours):** single-stream gain of +21–24% over no-spec; under concurrency ~99 tok/s at 8 streams and 141–146 tok/s at 12 (prod settings).
- **1M-context depth curve (NVFP4 build, our measurement):** 42.7 tok/s at 16K → 17.6 at 512K → 17.2 at a real 1.03M-token prompt — the curve flattens past 512K. Cold-prefill and prefix-cache timings at full depth included.
- **Reproduction:** the community author's concurrency-16 number re-derived
  within his own run-to-run variance on our cluster — his benchmark validated,
  and no speed win over it is claimed.

## Two honest negative results

1. **No sampler configuration reliably fixes long-context coherence** on the
   measured build: the collapse is not a fixed depth threshold, and one
   commonly suggested knob is simply blocked under spec-decode.
2. **Per-position acceptance decay — not bandwidth — drives the depth curve**,
   with the detailed analysis published rather than a "mystery slowdown" story.

## Why it matters for the lab

This repo is the lab's proof of deep runtime bring-up on non-standard hardware
(aarch64 + sm_121 is exactly the territory where "supported" and "works" are
different words), the working pattern for independent reproduction with clean
credit separation, and the source of the long-context and speculative-decoding
workload designs now in the workload library.

## What this does not establish

No generalization to datacenter Blackwell; no quality qualification of the
served model; single cluster, no external reproduction of *our* fp8 line yet.

---

Machine-readable claim registry: https://agmind.ai/claims.json · llms.txt: https://agmind.ai/llms.txt