CHTC GPU Lab: LLM Inference Example

A minimal example project for running Large Language Model (LLM) inference on CHTC GPU Lab at UW-Madison. Why This Project? CHTC&rsquo;s documentation is great, but setting up LLM inference involves several gotchas: GPU Lab nodes cannot directly access /staging via paths in submit files Version conflicts between container PyTorch and pip-installed packages Efficient environment management to avoid…