Disaggregated Quantization: Specializing LLM Prefill and Decode
Disaggregated Quantization specializes LLM inference by separating Prefill (compute-bound) and Decode (memory-bound) phases. By using specialized formats and weights, the technique boosts accuracy and enables offloading the Prefill model to SSD for long contexts.
Read article