llama.cpp

LLM inference engine implemented in pure C/C++, supports CPU and a variety of hardware, is lightweight and efficient