Conatus AI

Everything below was measured on a physical DGX Spark: NVIDIA GB10, compute capability 12.1 (sm_121), 48 SMs, 121.7 GiB unified memory, driver 580.159.03, CUDA 13.0, aarch64, Ubuntu 24.04.4. Model under test: unsloth/Qwen3.6-27B-NVFP4 with FP8 KV cache at 65,536 context, TP=1. Dated 2026-08-26; version-specific facts move fast.

The install path that works (no container)

uv venv --python 3.12 then uv pip install vllm --torch-backend=auto resolves and installs vLLM 0.27.1, FlashInfer 0.6.16.post3 and torch 2.13.0+cu130 from plain PyPI in about 32 seconds on the Spark's network. No nightly index, no NGC container required. The system Python is 3.14, which most ML wheels do not cover yet, so the pinned 3.12 venv is the whole trick.

The two landmines

First: FlashInfer's JIT invokes ninja from PATH. If you run the venv python by absolute path without activating the venv, ninja is not on PATH and the first NVFP4 GEMM build dies with FileNotFoundError: 'ninja'. Activate the venv or prepend its bin directory.

Second: there is no prebuilt sm_121 artifact for the CUTLASS FP4 path. FlashInferCutlassNvFp4LinearKernel JIT-builds the sm12x module on first start, so the first boot is slow and looks hung. It is not hung. Cold start to a green /health was 425 seconds; warm start 255 seconds (weights 98-115 s, torch.compile 8.5 s cached, FlashInfer autotune 21 s, CUDA graph capture 13-20 s).

What the engine resolves to on sm_121

On the 0.27.1 release the backend line reads decode_backend=flashinfer-native, prefill=torch.bfloat16, decode=torch.bfloat16, kv_cache_dtype=float8_e4m3fn, arch=sm121 with default cudagraph mode FULL_AND_PIECEWISE. XQA / TRT-LLM decode is gated off on the release for family 12x: requesting it logs TRTLLM attention is not supported on this platform and falls back to native (the gate opens on current nightlies). KV cache headroom after the 21.34 GiB model load: 83.65-86.23 GiB, about 2.5M tokens at this context length.

Correctness, not just boot

A server that boots can still generate garbage, so the cell was verified with needle recall at 2.5K/10K/16K/49K token depths, temperature 0, across four configurations (default, forced trtllm-attention request, enforce-eager, PIECEWISE graphs). All 16 probes returned the planted codeword exactly. One practical note for Qwen3.6 thinking models: send chat_template_kwargs: {"enable_thinking": false} when you need deterministic short answers, or the reasoning stream eats your max_tokens.

Primary sources

The full datapoint, including the release-vs-nightly XQA gate comparison with exact code cites, is posted in the FlashInfer SM121 support audit: flashinfer-ai/flashinfer#3170. Comparable measured cells for workstation Blackwell (sm_120) live in the serving matrix. To identify your own cell, run uvx --from git+https://github.com/jahnclawdmonet/blackwell-doctor blackwell-doctor; on GB10 it reports the unified memory correctly as of v0.1.1.

---

If a specific model, engine or quantization needs the same treatment on Blackwell hardware (sm_120 workstation or GB10), a single-cell verification with the exact commands, timings and raw logs is $59, written delivery, no call.