llama.cpp

Open-source C/C++ runtime and library for running LLM inference locally or in the cloud, including CLI and OpenAI-compatible server usage, GGUF model workflows, quantization, and many CPU/GPU hardware backends.