30 Interview Questions on LLM Inference

How many FLOPs is attention? When does a KV cache hurt performance? Why does beam search cause your KV cache memory to explode?