One-click server control
Start or stop llama-server
from the window or directly from the tray menu. LlamaTray builds the exact command for
you and shuts the server down cleanly when you close it.
LlamaTray
A lightweight system tray manager for local llama.cpp servers on Linux. Start, stop and monitor your models with one click.
PyQt6 · Python · works with llama-server from llama.cpp
✓ llama-server found: /home/user/.local/bin/llama-server ✓ Port 8080 is free. 🚀 Starting server: llama-server -m ~/models/qwen2.5-7b-instruct-q4_k_m.gguf --n-gpu-layers 99 --ctx-size 32768 --port 8080 ⏳ Server startup in progress... ✓ Server started successfully (PID: 48213)
Features
What LlamaTray does
LlamaTray sits in your system tray and wraps llama-server with a stable PyQt6 GUI — no terminal required for day-to-day use.
Single Model & Router modes
Point it at a single .gguf file, or
run llama-server as a router over an entire models directory and load or unload models on the fly.
Real-time resource monitor
CPU, RAM, GPU and VRAM usage as clean progress bars, refreshed every second — NVIDIA via
NVML/nvidia-smi, AMD via
rocm-smi/sysfs.
Run modes
Two ways to serve your models
Both modes generate a live command preview before you start, so you always know exactly what llama-server will run.
Single Model
Run one GGUF file directly. Select the model (or download it from HuggingFace), configure GPU layers, context size, port and optional mmproj, then start the server.
llama-server -m ~/models/qwen2.5-7b-instruct-q4_k_m.gguf --n-gpu-layers 99 --ctx-size 32768 --port 8080Router Mode
Expose an entire directory of GGUF models through llama-server's model router and load or
unload each model from the app via the /models API.
llama-server --models-dir ~/models --no-models-autoload --jinja --host 127.0.0.1 --port 8080 -ngl 99 --ctx-size 32768Requires a llama-server build with router support for Router Mode. Closing LlamaTray always terminates the server cleanly.
Also included
Other built-in tools
HuggingFace downloader
Search GGUF repositories inside the app, see exact file sizes and download models without leaving LlamaTray.
Profiles & advanced settings
Save named configurations with GPU layers, context size, port, sampler presets, mmproj and extra parameters.
llama.cpp manager
Scan your hardware, pick a backend (Vulkan, CUDA, ROCm, SYCL or CPU) and build from source or install a pre-built binary.
Installation
Install on Linux
Install from the AUR on Arch-based systems, or run the bundled installer script on other distributions.
Arch / AUR
yay -S llamatray-git
# or: paru -S llamatray-gitWorks on Arch, CachyOS and other AUR-based systems.
Ubuntu, Debian & Fedora
git clone https://github.com/DolbyDAX2/LlamaTray.git
cd LlamaTray
./install.shThe script sets up a venv, dependencies and an app-menu entry. GNOME needs the AppIndicator tray extension.
Open source on GitHub
LlamaTray is developed in the open under the MIT license.