Running even larger LLMs on small hardware

In a previous post [GM4] we ran gemma4 12B on a 16GB RAM budget; today we ll push the envelop and run deepseek-r1:14b-qwen-distill-q4_K_M on the same budget. While last time we got away by fiddling with the model definition, today we ll have to reconfigure the OS itself (in addition to fiddling with the model definition). While Continue reading Running even larger LLMs on small hardware