Llumnix moves a live LLM request instead of waiting for the queue

An annotated viewing of Alibaba's Llumnix OSDI talk: why append-only KV caches make live request migration possible, how virtual usage turns several scheduling goals into one load metric, and where the 2024 evidence stops.