Lecture 122:Disaggregated Quantization
Disaggregated Quantization specializes LLM inference by separating the compute-bound prefill phase from the memory-bound decode phase. This allows for tailored quantization formats and weights, boosting accuracy and speed.
Read article