Normalization Techniques in Transformer-Based LLMs: LayerNorm, RMSNorm, and Beyond

Deep dive into the evolution of normalization techniques in transformer-based LLMs, from the trusty LayerNorm to newer variants like RMSNorm, and even experimental tweaks.