Actually, That's Really Fast: Hybrid CPU-GPU Scheduling for LLM Inference — The ATSInfer Journey

The release of Qwen3.6-27B in April 2026 sparked a flurry of experiments in the llama.cpp community. While full GPU offload remains the default for high-end inference, practitioners with partial offload hardware (single RTX 4090, consumer GPUs, laptops with iGPUs) have pushed for smarter tensor placement strategies. The paper …