Map / Infrastructure / Inference & Compute / llama.cpp

llama.cpp

Open-source LLM inference in C/C++, plus the Llama app for your desktop

CuratedInference OptimizationLinuxMacWindows

I use this adds llama.cpp to your stack and finds people with similar stacks. Every mark feeds the counts on Pulse. Sign in to mark it.

No ratings yet

Who uses it

1 use this

1use it now

0tried it, dropped it

0want to try

People who use it, last 8 weeks

Every mark here is self-reported.

Ratings

Only from people who use it or tried it

No ratings yet

The first rating comes from someone who uses llama.cpp or tried it.

Use llama.cpp?

What it does

llama.cpp is an open-source C/C++ engine (MIT) for running large language and vision models locally on CPUs and GPUs, with backends for Apple Metal, CUDA, Vulkan, SYCL and others. It defines the GGUF model format, supports low-bit quantization, and ships a CLI and an OpenAI-compatible server with a web UI. Its team, now part of Hugging Face, also makes the Llama desktop app.

Alternatives on the map

All 58 alternatives

Inference & Compute, starting with the other Inference Optimization.

Compare with

People who use it also use

Open website