Early Breakthroughs in Transformer Quantization

In my previous blogpost , we discussed in detail the outlier problem in transformers. We realized that outliers are not just random noise - they encode important directional information that lets the model selectively skip updating token representations. But the same property makes transformers notoriously difficult to quantize. In this post, we …