Colibri: Running a 744 Billion Parameter LLM with 20 GB of RAM on FreeBSD

Modern frontier scale large language models (LLMs) are normally associated with clusters of very expensive GPUs and hundreds of gigabytes of memory. Yet recent advances in Mixture-of-Experts architectures and intelligent inference runtimes challenge this assumption. In this short article we explore how the 744 billion parameter GLM-5.2 model can be executed on an old dual-Xeon FreeBSD workstation…