Topic

Model Compression

All digests tagged Model Compression

Suggested route

Topic journey

Move from earlier context through the available material to the newest signals. This route is a reading aid, not a difficulty ranking.

Start here

Compression at the Edge — Chris Alexiuk, NVIDIA Aug 7, 2026 · 46:01 Model compression, particularly through advanced quantization like NVFP4 (a 4-bit float format), is critical for running massive LLMs on edge devices. Techniques allow shrinking models dramatically (e.g., GLM 5.2 from 1.5 TB to 250 GB) while maintaining performance.

Recent signals

Lecture 118: GSQ-RCO: Better Non-Uniform GGUF Quantization Oct 16, 2026 · 57:13 GSQ and RCO are complementary methods for accurate low-bit LLM quantization. GSQ uses Gumbel-Softmax sampling for stochastic optimization, while RCO uses Riemannian optimization to allocate precision across a model under a fixed size budget. GSQ-RCO: Better Non-Uniform GGUF Quantization Oct 14, 2026 · 57:20 GSQ and RCO are complementary methods for accurate low-bit LLM quantization. GSQ uses Gumbel-Softmax sampling for stochastic optimization, while RCO optimizes bit allocation under fixed size budgets, enabling state-of-the-art model compression. BF16 vs NVFP4 with Nemotron 3.5 Lightning Sep 14, 2026 · 9:20 Learn how NVFP4 compares to BF16 for Nemotron 3.5 Lightning. Discover how this low-precision format shrinks memory footprint and boosts inference speed without a major quality hit.

GPU MODE

Lecture 118: GSQ-RCO: Better Non-Uniform GGUF Quantization

GSQ and RCO are complementary methods for accurate low-bit LLM quantization. GSQ uses Gumbel-Softmax sampling for stochastic optimization, while RCO uses Riemannian optimization to allocate precision across a model under a fixed size budget.

Read article

GPU MODE

GSQ-RCO: Better Non-Uniform GGUF Quantization

GSQ and RCO are complementary methods for accurate low-bit LLM quantization. GSQ uses Gumbel-Softmax sampling for stochastic optimization, while RCO optimizes bit allocation under fixed size budgets, enabling state-of-the-art model compression.

Read article

NVIDIA Developer

BF16 vs NVFP4 with Nemotron 3.5 Lightning

Learn how NVFP4 compares to BF16 for Nemotron 3.5 Lightning. Discover how this low-precision format shrinks memory footprint and boosts inference speed without a major quality hit.

Read article

AI Engineer

Compression at the Edge — Chris Alexiuk, NVIDIA

Model compression, particularly through advanced quantization like NVFP4 (a 4-bit float format), is critical for running massive LLMs on edge devices. Techniques allow shrinking models dramatically (e.g., GLM 5.2 from 1.5 TB to 250 GB) while maintaining performance.

Read article