How Flash Attention Works

This is the third post in a series on LLM internals. Part 1 covered attention. Part 2 covered generation. This post: the Flash Attention algorithm and why it matters for long contexts.