Conatus AI

Serving LLMs on Blackwell (sm_120 workstation and server parts, sm_121 DGX Spark) throws a recognizable set of errors that are specific to the architecture and the runtime and backend combination, not to the model. This index maps the exact signature to a reproduction measured on RTX PRO 6000 Blackwell hardware, so a search for the literal error string lands on a concrete repro rather than a dead thread. It grows as new cells are measured.

FlashInfer sm_120 dynamic workspace OOM during profile_run

Signature: torch.OutOfMemoryError from allocate_sm120_dynamic_workspace, called from B12xMoEWrapper.__init__ inside GPUModelRunner.profile_run(), with --moe-backend=flashinfer_b12x on an NVFP4 MoE model.

What it is: not a hard sm_120 incompatibility. The b12x path reserves about 11.5 GiB more than the marlin backend before the KV cache is sized, so it OOMs 16 to 32 GiB Blackwell cards during profiling while starting on 96 GiB. --gpu-memory-utilization does not help, because the reservation happens before it applies. Full reproduction, the marlin comparison and the memory-profiler lines: measured on 96 GiB Blackwell. Upstream: vllm-project/vllm#49476.

CUTLASS FP4 GEMM allocates a private workspace per decode-sized call

Signature: repeated transient device allocations of roughly the shared workspace size on every mm_fp4 call with backend="cutlass", visible as allocator churn in the decode loop on sm_120.

What it is: measured on RTX PRO 6000 Blackwell, the extra per-call allocation is 34 MiB (against a 32 MiB shared workspace) and fires for batch sizes M from 1 to 256, then disappears at M of 384 and above, so it is confined to the decode and small-prefill regime and reproduces on FlashInfer 0.6.13. Upstream and the full per-M table: flashinfer-ai/flashinfer#4549.

SGLang: endless repetition and empty content on a compressed-tensors FP8 lm_head model

Signature: generation degenerates into a repeated phrase from the very first token (for example one city one city one city inside reasoning), content comes back empty, finish_reason: length on every request, and the load log contains Parameter lm_head.weight_scale not found in params_dict. Seen with mixed NVFP4 plus FP8 checkpoints whose quant config targets re:.*lm_head, for example unsloth/Qwen3.8-27B-NVFP4. The same checkpoint serves correctly on vLLM.

What it is: the FP8 lm_head.weight_scale is never applied, so raw FP8 weights are consumed as if they were bf16 and the logit ranking is corrupted. Fixed in sglang main by PR #35228 (2026-08-19), but the fix is in no released version up to and including v0.5.18, so release wheels keep the bug. Verified A/B on a single RTX PRO 6000 Blackwell (TP=1): one commit before the fix reproduces the repetition, current main answers correctly with zero scale warnings. Full table: sgl-project/sglang#34895. The matching before and after rows are in the serving matrix.

---

To identify which cell your environment is, run the open-source probe uvx blackwell-doctor (source); it prints your GPU, stack and a stable matrix key. If your exact cell is not yet in the index, it can be measured on reference RTX PRO 6000 Blackwell hardware and returned with startup, correctness, throughput, peak VRAM and raw logs: $59, written delivery, no call.