Learning Rate Warmup: Linear Warmup and Large Batch Training

Explains how learning rate warmup stabilizes early training by gradually increasing the learning rate, with theory, linear warmup. Read full notebook