Running Gemma 4 26B-A4B locally on 12GB VRAM

Google DeepMind's Gemma 4 family is out-truly open with Apache 2 licenses, multimodal with support for text, image, and audio, and efficient enough to run on-device. The 26B-A4B variant is a mixture-of-experts model with only 4B parameters active per token, making it an ideal candidate for consumer hardware. This post covers my setup running Gemma 4 26B-A4B on 12GB VRAM using mainline llama.cpp,…