← 全部应用

llama.cpp (CUDA 13)

ggml-org

Run local language models with NVIDIA CUDA 13 acceleration

第一次使用?配置 FlatPark →
AIWeb UIMITvb123594近 30 天新装136,100GitHub Star
The built-in llama.cpp browser chat interface (upstream screenshot)
The built-in llama.cpp browser chat interface (upstream screenshot)

关于

A community package of the unmodified official llama.cpp CUDA 13 build for local language model inference, an HTTP API and a browser chat interface. The included NVIDIA CUDA runtime libraries have their own proprietary license. Model downloads have their own licenses.

This edition requires an NVIDIA GPU and a host driver compatible with the packaged CUDA 13 runtime. Flatpak supplies the matching NVIDIA driver libraries through its GL extension. The package includes upstream's CUDA runtime libraries; a host CUDA toolkit is not required. CUDA needs access to NVIDIA device nodes, so this edition grants access to all devices.

The desktop entry starts llama-server in a terminal. Open your browser at the loopback address 127.0.0.1 on port 8082. Models downloaded by llama.cpp stay in this app's private persistent cache. With no model argument, the server starts in router mode for cached models. Download a model first using the CLI, for example: flatpak run --command=llama-cli io.github.ggml_org.LlamaCpp.CUDA13 -hf bartowski/SmolLM2-135M-Instruct-GGUF:Q4_K_M

Command line tools are available with flatpak run --command=llama-cli io.github.ggml_org.LlamaCpp.CUDA13, and similarly llama, llama-bench and llama-quantize. Pass server model and port options after flatpak run io.github.ggml_org.LlamaCpp.CUDA13. The CPU, Vulkan and CUDA 13 editions use different default ports and private caches.

Host model directories are optional. Grant only a directory you need: flatpak override --user --filesystem=/path/to/models:ro io.github.ggml_org.LlamaCpp.CUDA13. Then pass -m /path/to/models/model.gguf. To remove overrides: flatpak override --user --reset io.github.ggml_org.LlamaCpp.CUDA13.

Python model conversion and workflows requiring a host toolchain are not included.

详情

开发者
ggml-org
许可证
MIT
运行时
Freedesktop 26.08
架构
x86_64
版本
b12359 · 2026-11-13
网站
主页

权限

2 项需确认
  • 全部设备(含摄像头、USB)

  • 网络访问

    可访问互联网和本地网络

打包方式

原样

按厂商发布的原样运行,没有添加或修改任何东西。

这些级别是什么意思 →