drowzeys/DeepSeek-V4-Flash-DSpark-NVFP4-KV-1.5M-CTX-2xDGX-Spark

DeepSeek-V4-Flash-DSpark · NVFP4-KV · 1.5M CTX · 3M Pool · C=12 · 2× DGX Spark

A serving recipe / configuration card for running deepseek-ai/DeepSeek-V4-Flash-DSpark with DSpark speculative decoding and an nvfp4_ds_mla 4-bit KV cache on 2× NVIDIA DGX Spark (GB10, sm_121a) at TP=2, for a 1.5M-token context window.

This card does not re-host model weights. NVFP4 here is a runtime KV-cache setting — the weights are the unmodified base model. Pull the base model from deepseek-ai and apply this recipe (below). Full sources, benchmark harness, and the vLLM 0.24.0 port fix live in the GitHub repo: https://github.com/drowzeys/keys-vLLm-0.24.0-Optimized-DeepSeekV4-Flash-DSpark-NVFP4-KV-1.5M-CTX-3M-Pool-C-12-on-2-DGX-Spark

Headline

• 1.5M context window backed by a 3,231,736-token KV pool (util 0.85).
• Coherent needle retrieval validated to 543,994 tokens — no garble.
• C=12 concurrency at ~255 tok/s aggregate; 61–67% DSpark acceptance.
• Runs on two DGX Spark boxes over RoCE/IB.

Standing config

Parameter	Value
KV cache	nvfp4_ds_mla (4-bit MLA KV)
Context window	1,500,000 (max_model_len)
KV pool	3,231,736 tokens (21.25 GiB @ util 0.85)
max_num_seqs	12
gpu_memory_utilization	0.85
Speculative	DSpark, MTP_NUM_TOKENS=5
VLLM_USE_B12X_WO_PROJECTION	1 (high-acceptance-under-concurrency profile)
Topology	2× DGX Spark GB10, TP=2

Native trained context is 1,048,576; 1M–1.5M is RoPE-extended (VLLM_ALLOW_LONG_MAX_MODEL_LEN=1). Validated-coherent to 512k. See RESULTS.md.

Download & serve (two commands)

# 1) pull the (unmodified) base model
hf download deepseek-ai/DeepSeek-V4-Flash-DSpark --local-dir ./DeepSeek-V4-Flash-DSpark

# 2) build the GB10 serving image + launch with this recipe
#    (image build + launcher in the GitHub repo; run worker then head)
ssh <worker> '~/dspark-nvfp4-serve.sh 1'
~/dspark-nvfp4-serve.sh 0     # serves an OpenAI-compatible API on :8000

dspark-nvfp4-serve.sh (included here) carries the exact serve args, the NVFP4/DSpark/garble-fix env, and the 1.5M/util-0.85/seqs-12/WO=1 profile. Edit the fabric vars (MASTER, IF, HCA, GID) and model path for your cluster.

Files

• dspark-nvfp4-serve.sh — the serving launcher (this exact config)
• RESULTS.md — full benchmarks: concurrency, context, and long-context (6k→512k) sweeps
• CREDITS.md — attribution

Credits

Special thanks to tonyd2wild (the nvfp4_ds_mla 1M stage recipe), MiaAI-Lab (DSpark DGX Spark packaging), Rafael Caricio (DSpark vLLM integration), and the vLLM project. Transplant kernels credited to aidendle94 (compiled flashinfer sparse-MLA + DeepGEMM), CosmicRaisins (Triton sparse-MLA), and hazyumps (GB10 indexer). Model: deepseek-ai/DeepSeek-V4-Flash-DSpark. Full details in CREDITS.md.

Support / Donations

If this work is useful, donations help fund more open GB10 / DGX Spark serving research:

• Solana: drkeys.sol

Thank you 🙏