Outline of LLM acceleration

Summary Methods There are two main methods to acclerate LLM and another tricky methods low-rank: reduce dimension of matrix block: compute matrix with block trick: update model structure or change training process Reference xformers: collection of optimized transformers unsloth triton Liger-Kernel Flash-attention Megatron-LM from NVIDIA huggingface accelerate ColossalAI awesome LLM system…