Glossary
What is SmoothQuant? Definition and meaning
Moves quantization difficulty from activations to weights with a mathematically equivalent scaling, enabling 8-bit weights and activations.
Definition
SmoothQuant: Moves quantization difficulty from activations to weights with a mathematically equivalent scaling, enabling 8-bit weights and activations.
In short: Quantizing activations as well as weights.
When it is used
Throughput-oriented serving with 8-bit arithmetic.
Learn more
The full guide to SmoothQuant explains how it works, when to use it and how it compares with the alternatives, with examples. The glossary lists every term on the site from A to Z.
Previous term: smolagents · Next term: snnTorch