NixNan: A tool for FP exception tracking in NVIDIA GPUs
Summary
NixNan is a specialized binary instrumentation tool designed to track floating-point (FP) exceptions, such as NaNs and Infinities, within NVIDIA GPUs. Given that high-performance computing (HPC) and machine learning (ML) increasingly rely on low-precision formats (e.g., FP16, BF16) and specialized hardware (Tensor Cores), numerical stability becomes critical. NixNan operates at the Streaming Assembly (SASS) level, allowing engineers to observe exceptions per instruction, per precision, and even track if the exceptions are written to memory, providing deep visibility into numerical failures that standard source instrumentation cannot achieve.
Key takeaways
-
The Challenge of Floating Point in GPUs
Floating point arithmetic is inherently complex, and when low precision is used or specialized hardware is involved, numerical issues (NaNs, Infinities) can corrupt entire training runs. Standard source instrumentation often fails to capture these deep-level errors.
-
NixNan's Functionality
2:00
NixNan uses binary instrumentation (built on NVBit) to observe the execution of CUDA kernels at the SASS level. It tracks exceptions for various precisions (FP16, BF16, FP32, FP64) and records details like the number of occurrences and whether the results are written to memory.
-
Debugging Complex Failures
4:10
The tool allows for advanced debugging techniques, including histogramming of exponents to identify computational ranges prone to failure, and tracking exception flow through complex routines like LU solvers and Tensor Core operations.
-
Hardware Limitations
3:20
Unlike CPUs, NVIDIA GPUs do not natively trap NaNs, making external, deep-level observation tools like NixNan essential for diagnosing numerical stability issues.
More digests on Floating Point Arithmetic
Technical details
-
Floating Point Arithmetic
150s
Floating point numbers pack a large dynamic range into few bits. Lower precision is sought for energy efficiency and speed, but this increases the likelihood of quirky behaviors, rounding errors, overflows, and NaN/Infinity generation.
-
NixNan Architecture
320s
NixNan is built on NVBit, a binary instrumentation tool. It instruments CUDA programs by inserting logic into the execution flow at the SASS level. This allows observation without needing access to the original PTX or CUDA C++ source code.
-
Exception Tracking
360s
The tool tracks exceptions (NaNs, Infinities) per instruction, per precision (e.g., FP16, BF16, FP32, FP64). It can also determine if the exception result is written to memory using the `STG` instruction, indicating a potential data leak.
-
Tensor Core Operations
470s
The tool successfully instruments complex, largely undocumented hardware units like Tensor Cores, which perform matrix multiplications. This demonstrates the ability to observe numerical issues even in highly optimized, fixed-function hardware blocks.
Mentioned resources
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.