Local LLMs on Titan RTX

After getting my hands on a Titan RTX with 24GB of VRAM, [shoutout to the un-imaginatively named PC Server and Parts ] I decided to move past hosted AI services and see what it could do in a Lenovo P920 with Rocky Linux. The Stack The goal was a clean, containerized setup that could leverage the GPU without cluttering the host OS: OS: Rocky Linux 9 Engine: Ollama running in Podman Frontend: Open…