llama.cpp (Vulkan)
ggml-org
Run local language models with Vulkan GPU acceleration
Copied! Paste it into a terminal and press Enter to install.

About
A community package of the unmodified official llama.cpp Vulkan build for local language model inference, an HTTP API and a browser chat interface. Model downloads have their own licenses.
The desktop entry starts llama-server in a terminal. Open http://127.0.0.1:8081 in your browser. Models downloaded by llama.cpp stay in this app's private persistent cache. With no model argument, the server starts in router mode for cached models. Download a model first using the CLI, for example: flatpak run --command=llama-cli io.github.ggml_org.LlamaCpp.VULKAN -hf bartowski/SmolLM2-135M-Instruct-GGUF:Q4_K_M
Command line tools are available with flatpak run --command=llama-cli io.github.ggml_org.LlamaCpp.VULKAN, and similarly llama, llama-bench and llama-quantize. Pass server model and port options after flatpak run io.github.ggml_org.LlamaCpp.VULKAN. The CPU and Vulkan editions use different default ports and private caches.
Host model directories are optional. Grant only a directory you need: flatpak override --user --filesystem=/path/to/models:ro io.github.ggml_org.LlamaCpp.VULKAN. Then pass -m /path/to/models/model.gguf. To remove overrides: flatpak override --user --reset io.github.ggml_org.LlamaCpp.VULKAN.
Python model conversion and workflows requiring a host toolchain are not included.
Details
- Developer
- ggml-org
- License
- MIT
- Runtime
- Freedesktop 26.08
- Architectures
- x86_64
- Version
- b12359 · 2026-11-13
- Website
- Homepage
- Support
- Help & support
- Source
- llama.cpp (Vulkan)
- Packaging
- flatpark/registry
Permissions
1 to reviewNetwork access
Can reach the internet and local network
GPU acceleration
Packaging
UnmodifiedRuns the vendor's build as shipped — nothing added or changed.
What the levels mean →