Two Bugs That Hid in Plain Sight: A vLLM Debugging Detective Story — Asaf Gardin & Yuval Belfer
Two critical bugs in stateful inference (vLLM/Mamba) were found by starving the GPU and comparing logprobs. The failures were silent, requiring deep kernel debugging and type fixes.
Read article