Small Language Models

This is part of a series about trying to set up a Small Language Model locally on inferior hardware. Check PSU Check Motherboard, Case dimensions, airflow Check if PCI lane limitations are an issue PCIe lane limitations aren’t an issue The first slot runs at PCIe 4.0 x16 The second slot runs at PCIe 4.0 x4 For LLM workloads (LLAMA.cpp, GPTQ, ExLlamaV2, TensorRT-LLM, etc.), this is completely fine.…