Cross Entropy Loss of Triton

Cross Entropy idea of fast cross entropy Based on previous knowledge on fast cross-entropy, realizing it by triton doesn’t spend too much. There are only 1e-7 difference between Pytorch and my Triton kernel, But the speed is significantly lower. Liger-Kernel Getting advise on gpumode discord group, I can read how it realized in other great Triton kernel repository. Liger-Kernel is the best option,…