Conatus AI

Question. The reporter, carstenkreissl, saw Xid 13, Out Of Range Address and CUDA illegal memory access under sustained load on an RTX PRO 5000 Blackwell (SM120), serving an FP8 model with vLLM v0.28.0 and FlashInfer 0.6.16.post3. Changing the attention backend still faulted. Disabling FlashInferFP8ScaledMMLinearKernel selected CutlassFP8ScaledMMLinearKernel and ran clean with CUDA graphs on; --enforce-eager also ran clean. Which runner inside FlashInfer did the failing configuration select?

Result. FlashInfer's autotuner selected the cuDNN bmm_fp8 runner for the reporter's prefill bucket. On SM120 with cuDNN below 9.23.1, FlashInfer 0.6.18.post1 gates that runner out of the auto candidates. The reporter confirmed the gate in the startup log and cache, completed a clean run with the FlashInfer kernel and autotune still enabled, and adopted the package swap. The measurements support the cuDNN runner as the faulting path; they do not establish whether its workspace resize caused the fault.

Environment

GPU        : NVIDIA RTX PRO 5000 72GB Blackwell, SM120
container  : Ubuntu 24.04.3 LTS, x86_64
host       : Ubuntu 24.04.4 LTS, kernel 6.8.0-139-generic
driver     : 595.84
image      : vllm/vllm-openai:v0.28.0
PyTorch    : 2.13.0+cu130
CUDA       : build 13.0, runtime 13.0.88
FlashInfer : python 0.6.16.post3, cubin 0.6.16.post3,
             jit-cache 0.6.16.post3+cu130
model      : mistralai/Ministral-3-14B-Instruct-2512
config     : quantization=fp8, kv_cache_dtype=auto,
             cudagraph_mode=FULL_AND_PIECEWISE,
             max_num_batched_tokens=8192

Environment from the issue body. The reporter's cache later recorded cuDNN 9.20.0 (cudnn_version: 92000) and frontend 1.27.0.

What was read

The code analysis is based on reading source. We ran nothing on this issue; all measurements below are the reporter's. The source references are the release tags and commits recorded in our thread comments.

At vLLM v0.28.0, vllm/model_executor/kernels/linear/scaled_mm/flashinfer.py routes FlashInferFP8ScaledMMLinearKernel.apply_scaled_mm through flashinfer_scaled_fp8_mm in vllm/utils/flashinfer.py to flashinfer.bmm_fp8(..., backend="auto"). The kernel file and both bmm_fp8 call sites are unchanged at the FlashInfer version bump, commit 7ab29234890b (#54313).

At FlashInfer 0.6.16.post3, flashinfer/gemm/gemm_base.py builds the SM120 auto list in _heuristic_func_bmm_fp8: CUTLASS sm12x, cuBLAS, then cuDNN when available. bmm_fp8 allocates the shared bmm_fp8_workspace at 32 MiB and calls fp8_gemm_sm100, then tuner.choose_one("fp8_gemm", ...). In flashinfer/autotuner/autotuner.py, no tuning data means runners[0], CUTLASS sm12x.

vLLM v0.28.0's vllm/model_executor/warmup/kernel_warmup.py runs FlashInfer autotune on SM120 unless explicitly disabled. Its dummy run uses max_num_batched_tokens. FlashInfer 0.6.16.post3's flashinfer/fused_moe/utils.py maps the reporter's 405 to 482 new-token prefills to M = 512 through map_to_hybrid_bucket_uncapped.

In flashinfer/gemm/gemm_base.py at 0.6.16.post3, CudnnFp8GemmRunner reaches execute_cudnn_gemm_fp8_graph. When a plan needs more workspace, it calls workspace.resize_(); CUTLASS and cuBLAS do not resize this buffer. Moving it can leave previously captured CUDA graphs with an old address. This is a source-level mechanism, not a resize observed on the reporter's box. Instrumenting it requires logging from process start, since the buffer can grow during warmup and remain grown throughout a failing load run.

What the reporter measured

Each user question produces 2 to 3 completions. Counts below are faults per question. The reporter used the same machine and question pool, with a client-side concurrency limit of 6; the later runs identify a 122-question pool. CUDA graphs remained enabled in every arm below.

Before the swap, cuDNN won 67 of 84 tuned shapes and all four M = 512 entries. After the swap, cuDNN won none, cuBLAS won 65 and CUTLASS won 19; all four M = 512 entries were cuBLAS. cuDNN remained importable with the same cache metadata. The original cache was reconstructed at a later startup without load; caches from the crashing containers were lost.

The reporter's caveat: at roughly one fault per 400 to 500 questions, a single clean hour is about 13 percent likely by luck. What carries the conclusion is three clean arms on one machine where the cuDNN runner never executes: kernel disabled, tuning skipped and cuDNN gated out. Faults occurred only in the arm where it did. This does not prove the workspace resize was the cause. The swap also upgraded nvidia-nccl-cu13 to 2.29.7, whose effect was not isolated. An index rebuild and unrelated engine changes prevent a latency comparison across these runs.

The version boundary

FlashInfer #4165, commit 3e96dfa35ee3, adds the gate in flashinfer/gemm/gemm_base.py. It is absent in 0.6.16.post3 and 0.6.17; 0.6.18 is the first release containing it, and 0.6.18.post1 carries it too. On SM12x, cuDNN below 9.23.1 lacks the override-shape path, so the auto heuristic excludes the cuDNN bmm_fp8 runner. An old cache entry naming that absent runner is ignored. The reporter's cuDNN 9.20.0 meets this condition.

As of the thread's 2026-09-07 follow-up, FlashInfer #4666, commit ec45ed3072bc, was merged to main and in no release. It replaces the shared-workspace resize_() sites with _gemm_workspace_at_least(), raises the default to 40 MiB and refuses plan builds during capture. It also covers the bf16 and fp4 cuDNN paths and cuDNN at or above 9.23.1, where the FP8 gate does not apply. The gate and the workspace fix are separate changes.

The v0.28.0 image pins FlashInfer 0.6.16.post3 in requirements/cuda.txt and ships a jit-cache package. In flashinfer/jit/env.py at 0.6.18.post1, mismatched cubin or jit-cache versions raise at import. The reporter moved all three packages together, overriding the image's tested pin:

flashinfer-python==0.6.18.post1
flashinfer-cubin==0.6.18.post1
flashinfer-jit-cache==0.6.18.post1+cu130

Python came from PyPI, cubin from https://flashinfer.ai/whl/, and jit-cache from https://flashinfer.ai/whl/cu130/ for this cu130 image. This is the reporter's tested combination; it remains an override of vLLM's release pin.

How to check your own box

Locate the file named by Using FlashInfer autotune cache file: in the startup log. Its fp8_gemm keys name the winning runner. CudnnFp8GemmRunner means cuDNN was selected; inspect the M = 512 entries for this prefill bucket. Cache data shows selection, not whether a workspace resize or a fault happened.

For a check after the package swap, leave VLLM_DISABLED_KERNELS empty, unset VLLM_FLASHINFER_AUTOTUNE_SKIP_OPS so tuning runs, and point VLLM_FLASHINFER_AUTOTUNE_CACHE_DIR at a fresh empty directory. The fp8_gemm entries should name only CutlassFp8GemmRunner and CublasFp8GemmRunner for this configuration. When testing the tuning-skipped arm instead, use VLLM_FLASHINFER_AUTOTUNE_SKIP_OPS=fp8_gemm with an empty cache directory: skipping tuning alone still permits a loaded cache to choose the runner at inference.

Grep the complete startup log for this exact line prefix:

Skipping cuDNN in bmm_fp8 auto candidates on SM12x

This one-shot warning fires at the first eligible FP8 GEMM, during the memory-profiling forward at startup. Its presence confirms the gate. Its absence is inconclusive: cuDNN may never have reached the candidate list. Check the fresh cache as well.

Sources

• vLLM #55571, issue body and original experiments, 2026-09-06, carstenkreissl.
• Extended CUTLASS soak, 2026-09-06, carstenkreissl.
• Initial source analysis, 2026-09-06, jahnclawdmonet. Its upgrade conclusion is corrected in the version-boundary reply below.
• Runner cache and reconstruction caveat, 2026-09-06, carstenkreissl.
• Version-boundary correction, 2026-09-06, jahnclawdmonet.
• Tuning-skipped run, 2026-09-07, carstenkreissl.
• Pin comparison and matched packages, 2026-09-07, jahnclawdmonet.
• Gate confirmation, clean run and adopted swap, 2026-09-07, carstenkreissl.

---

To fingerprint your own environment and get its matrix key, run the open-source probe: uvx --from git+https://github.com/jahnclawdmonet/blackwell-doctor blackwell-doctor (source). It reports your Blackwell GPU, serving stack and the exact cell you are running, with no network calls.

This page is the shape of a single-cell serving verification: one public model and revision, one runtime, one quantization, one topology, one load point, measured on RTX PRO 6000 Blackwell and handed back with the exact command, environment and raw logs. If you have a Blackwell serving combination you need checked (does it start, does it stay correct, what does it cost), that fixed-scope check is $59, written delivery, no call.