# DeepSeek-V4-Flash 0731 on two DGX Sparks: the recipe, with its traps marked

> The exact serving recipe behind our published GB10 numbers: weights and image pinned, the head-first launch order that avoids a TCPStore hang, the MoE backend flag auto-selection skips, what to monitor when NVML goes silent — and the numbers to expect.

- Published: 2026-08-16
- Evidence level: lab_single_run
- Funding: Self-funded internal research
- Canonical: https://agmind.ai/reports/deepseek-v4-flash-0731-dgx-spark-recipe/

This is the runbook behind our published DeepSeek-V4-Flash-0731 numbers on a
pair of DGX Sparks (GB10, TP=2 over 200G RoCE) — every step the way we
actually run it, with the traps that cost us time marked as traps. The
authoritative, versioned copy lives in the
[public repository](https://github.com/botAGI/dspark-0731-gb10) with the run
manifest and raw outputs; this page is the annotated tour.

## The five steps

**1. Weights on BOTH nodes** — ~156 GiB on disk per node . We
benchmarked the community FP8 checkpoint pinned by revision; the official
checkpoint is noted in the repo but is not what our numbers describe. Pin
the revision — `main` moves.

**2. Runtime image by digest, on both nodes.** Tags move; the digest in the
repo does not. Every published number names the digest it ran under —
that is what makes a disagreement diagnosable later.

**3. HEAD FIRST, then the peer.** Rank 0 hosts the TCPStore; a peer that
connects before the head is listening hangs on a broken pipe with no useful
error. Launch the head, wait until `ss -tln` shows the store port listening,
only then launch the peer. This ordering bug reads obvious written down and
cost us real hours live.

**4. Ready when `/v1/models` answers.** Weight load takes ≈6–7 minutes
; the fabric settings (head/peer IPs, HCA, GID index) are environment
variables at the top of the launch script.

**5. Verify the MoE backend actually took effect.** The engine ships a
kernel purpose-built for this model's MXFP4 experts on GB10 and deliberately
excludes it from `auto`:

```
docker logs vllm_dsv4 2>&1 | grep "Mxfp4 MoE backend"
# want: Using 'B12X_MXFP4'     not: Using 'DEEPGEMM_MXFP4'
```

Skipping this check is the single most common reason a published Spark
number lands in the high fifties instead of the high sixties: `auto` gave us
59.7 tok/s client decode where `flashinfer_b12x` gave 67.6 ,
a 9–12% verification-step-rate gain measured engine-side .

## The numbers to expect

Final configuration, medians over repeated requests, from the
[public results](https://github.com/botAGI/dspark-0731-gb10#results) :

| Cell | tok/s |
| --- | --- |
| card-like short output, c=1 | 67.6 |
| code profile, c=1 | 65.5 |
| heavy 4K-context prose, c=1 | 43.7 |
| code profile, c=12 aggregate | 260.4 |

The profile spread is larger than most configuration changes — quote a
number without its profile and it is unfalsifiable. Why the figures
circulating in forums disagree with each other is
[its own page](https://agmind.ai/reports/dspark-speed-numbers-disagree/).

**Count tokens from `usage.completion_tokens`, never from SSE event count.**
Under speculative decoding one event can carry several accepted tokens;
counting events silently inflates throughput.

## The monitoring trap

On GB10 unified memory, `dcgm-exporter` does not work at all and NVML
returns N/A on a large share of queries — your Grafana will be green and
empty. We parse `nvidia-smi` with a custom textfile collector into
node-exporter metrics; the write-up is
[on Habr, in Russian](https://habr.com/ru/articles/1030802/). Budget for
this: a serving box you cannot observe is a box you cannot operate.

## Honesty section

- Everything here is `lab_single_run`: single passes with warmups, one pair
  of units, outside our three-repeat methodology. Error bars on this
  hardware class are wide — identical aggregate runs varied 18–24%
   — so treat single-decimal precision from any source, including
  ours, as optimistic.
- Our own first measurement pass was client-side-biased and we marked it
  superseded in the repo rather than deleting it; the engine-side series is
  the one to quote.
- Found a number that disagrees?
  [Submit it](https://github.com/botAGI/agmind-lab/issues/new?template=conflicting-number.md) —
  the configuration delta behind a disagreement is usually more informative
  than either figure.

Related: [the 1M-context depth curve](https://agmind.ai/reports/dspark-speculative-1m-dgx-spark/) ·
[the MoE backend report](https://agmind.ai/reports/dsv4-flash-moe-backend-dgx-spark/) ·
[is the Spark worth it at all](https://agmind.ai/reports/dgx-spark-worth-it/).

---

Machine-readable claim registry: https://agmind.ai/claims.json · llms.txt: https://agmind.ai/llms.txt