Local AI 301: Quantization
This talk details advanced quantization techniques (QAD, I-Matrix) and mixed-precision inference using formats like MXFP4. Learn how to optimize LLMs for speed and size on modern GPUs.
Read articleTopic
Independent digests of GGUF talks — key takeaways, technical details, and related reading.
Suggested route
Move from earlier context through the available material to the newest signals. This route is a reading aid, not a difficulty ranking.
This talk details advanced quantization techniques (QAD, I-Matrix) and mixed-precision inference using formats like MXFP4. Learn how to optimize LLMs for speed and size on modern GPUs.
Read article