Run Free LLMs at Scale: LiteLLM Gateway with Groq, NVIDIA NIM, OpenRouter, and Local vLLM

Introduction Running large language models is increasingly affordable — but &ldquo;affordable&rdquo; rarely means &ldquo;free, all the time, for every request.&rdquo; Cloud providers each come with their own rate limits, daily quotas, and occasional model deprecations. Local hardware is fast and private, but not always available (DGX Spark powered down, model being updated, VRAM needed elsewhere).…