vLLM at 256K context on DGX Spark: the configuration hunt, and the NVFP4 path that was broken in mainline

This is the English companion to the lab’s Russian long-read on Habr: getting a 35B-class MoE to serve a 256K context window on a single DGX Spark, and what we found broken along the way. That article is the primary source for every figure here — the numbers below are quotes from it, not lab registry claims.

Read the version pin first. Everything on this page describes one moment in a moving target: vLLM 0.15-era mainline with PyTorch 2.10, FlashInfer 0.6.8, driver 580.142, CUDA 13.0, DGX OS 7.5.0, measured in May 2026. Mainline moves weekly. What was broken then may be fixed by the time you read this, and what worked then may have regressed — treat this page as dated evidence with its pins attached, and check the linked upstream issues before repeating our conclusions.

The machine and the goal

One DGX Spark: GB10, Blackwell SM_121, 128 GiB of unified LPDDR5x at 273 GB/s . The goal was a genuinely long window — 262,144 tokens — for a Qwen3.6-35B-A3B-class MoE, on hardware whose compute architecture most open-source kernels had never been compiled for.

The configuration hunt

What refused to run, in the order we hit it:

• FlashInfer FP8 attention rejected the chip outright — the kernel supports sm120, and SM_121 got illegal synchronization errors instead of a fallback.
• CUTLASS/FA3 FP4 paths carry a heuristic that requires a minor compute-capability of zero, which excludes GB10 by construction.
• Triton fallback ran and stayed stable, at a cost of roughly 10–30% against FlashInfer .
• MoE backend flags needed hand-setting: VLLM_USE_FLASHINFER_MOE_FP4=0 because the autotuner skipped ten of seventeen tactics as unsupported , while VLLM_USE_FLASHINFER_MOE_FP8=1 worked on SM_121 with FlashInfer 0.6.x.
• Full CUDA-graph capture hung on vLLM 0.15-era builds; cudagraph_mode=piecewise is the workaround.
• Driver 580.142 is the pin. The 590.48.01 driver leaked unified memory until 80 of 128 GB were consumed with nothing visible in AnonPages or Slab , and 595.58.03 broke NVFP4 with a cuTensorMapEncodeTiled illegal memory access. If your Grafana looks fine while this happens, see why GB10 monitoring lies.

Why NVFP4 was broken in the mainline build we tested

The pain query that probably brought you here. In the mainline build we pinned, NVFP4 kernels were compiled for compute_120f, while the native NVFP4 instructions exist only in compute_120a and compute_121a. On SM_121 that meant quantized weights were unpacked in software, bit manipulation in the shader with the tensor cores idle. The log fingerprint was [AutoTuner]: Skipped 10 unsupported tactic(s) for trtllm::fused_moe::gemm2.

The measured consequence: the quantization with half the memory footprint decoded slower than FP8 — 40.9 against 51.2 tok/s single-stream at the 260K configuration . Half the bytes, more time: the unpacking overhead ate the bandwidth win.

To be precise about tense: NVFP4 was broken in that build, on that chip, at that date. This is a compilation-target gap, not a law of the hardware — the upstream tracker was flashinfer issue #3170, the “DGX Spark SM_121 Audit”, with seventeen items open when the article shipped . Verify against current mainline before deciding anything.

What worked

The configuration that held 260K and went fastest was nothing like stock: vLLM built from source with seven local patches, FlashInfer 0.6.8 plus PRs #2520 and #2702, the community AEON-7/Qwen3.6-35B-A3B-heretic-NVFP4 checkpoint, and the DFlash block-diffusion drafter at fifteen speculative tokens:

--max-model-len 262144
--max-num-seqs 128
--max-num-batched-tokens 65536
--gpu-memory-utilization 0.85
--enable-chunked-prefill --enable-prefix-caching
--speculative-config '{"method":"dflash","num_speculative_tokens":15}'
--attention-backend flash_attn

The trap in the fast path: the heretic fine-tune breaks tool calling (its issue #4, unfixed at the time). For agent and MCP serving the article’s recommendation was vanilla Qwen/Qwen3.6-35B-A3B-FP8 on stock vLLM — slower, but function calls come back intact.

Measured behavior at 256K

Quoted from the article’s single-box runs :

Warm time-to-first-token was around 100 ms including drafter compilation . The most useful surprise is how cheap the window itself was: growing max-model-len from 65K to 260K cost about two and a half percent of single-stream decode , while a single user’s active context rarely exceeded 30K of the 260K window . The speculative numbers swing hard by workload — DFlash acceptance was 17.8% overall but 78% at position zero on math and code , which spread decode between 117 tok/s on math and 45–57 tok/s on business chat . And the ceiling is memory, not marketing: at 273 GB/s the article’s roofline puts effective speculative decode near 320 tok/s , regardless of what the FP4 petaflop figure on the box implies.

Honesty section

• Everything here is lab_single_run: one lab, one box, single passes. None of these figures are registry claims — they are quotes from our own article, carrying its caveats.
• The version pin at the top is the whole point. A “broken” verdict without its build fingerprint is noise; with it, it is a reproducible bug report.
• The two-node serving recipe with its own traps is a separate page, and why published Spark numbers disagree with each other is its own investigation.