Compression at the Edge — Chris Alexiuk, NVIDIA
Model compression, particularly through advanced quantization like NVFP4 (a 4-bit float format), is critical for running massive LLMs on edge devices. Techniques allow shrinking models dramatically (e.g., GLM 5.2 from 1.5 TB to 250 GB) while maintaining performance.
Read article