Hands-on LLM Inference Bake-Off

In this post: LLM self-hosting and serving with vLLM and SGLang (8 min read) As the ecosystem around open-weight LLMs expands, so do its options for inference serving. Open-source software enables self-hosting these models on varied hardware, from outright GPU clusters to workstations, laptops and even smartphones. This post shares a first-hand account of trying to get these models to speed, from…