llama.cpp

llama.cpp is a LLM inference framework written in C++ and primarily meant for edge computers like laptops. It is built on top of GGML, a low-level tensor library and uses its own GGUF file format for models.

Install

Install using brew:

$ brew install llama.cpp

Run a LLM

Download and run a HuggingFace model:

$ llama-cli -hf ggml-org/gemma-3-1b-it-GGUF