Usage

Day-to-day LlamaTray workflows: the tray menu, Single Model and Router modes, and the built-in llama.cpp manager.

The interface

LlamaTray is a single window with three tabs plus a bottom bar:

• Main — model selection (or models directory), start/stop controls, command preview, resource monitor.
• Settings — run mode, advanced configuration, mmproj, language and application preferences.
• Profiles — save, load, update and delete named configurations.

The bottom bar hosts the llama.cpp manager entry point.

System tray menu

When a system tray is available, the tray icon offers:

The tray icon reflects the server state:

server stopped

server running

Minimizing to the tray can be enabled in the settings and is remembered between sessions.

Single Model mode

1. Select Single Model under Settings (default).
2. Choose a .gguf file on the Main tab — or use Download from HF.
3. Set GPU layers, context size, port, sampler preset, optional mmproj file and extra parameters.
4. Check the live command preview and click Start Server.

The generated command looks like:

llama-server -m /path/to/model.gguf --n-gpu-layers 99 --ctx-size 32768 --port 8080

Extra parameters you add are appended to the command as-is, so flags such as -t 8 or --flash-attn work when your build supports them.

Router Mode

1. Select Router Mode under Settings.
2. Choose the directory containing your GGUF files (required).
3. Toggle Auto-load models and Jinja, then set GPU layers, context size and port.
4. Start the server — the active model list is refreshed automatically from GET /models.
5. Use Load and Unload beside each model. The state column shows Loaded, Unloaded, Loading or Unknown.

The generated command looks like:

llama-server --models-dir /path/to/models --no-models-autoload --jinja \
  --host 127.0.0.1 --port 8080 -ngl 99 --ctx-size 32768

Download from HF stays available in Router Mode and downloads into the configured models directory by default.

Web UI

In either mode, Open Web UI opens llama.cpp’s built-in web interface (on http://127.0.0.1:<port>) in your default browser once the server is running.

Stopping the server

• Click Stop Server, or simply close LlamaTray — both terminate the server cleanly.
• LlamaTray first asks the server to exit over its HTTP API, then verifies the process and port are actually freed, forcing cleanup if necessary.

llama.cpp manager (build & install)

If you don’t have llama-server yet, open llama.cpp Manager from the bottom bar:

1. Hardware scan — runs a pre-flight check (nvidia-smi, rocminfo, vulkaninfo, clinfo, lspci) and shows a recommendation banner (e.g. Vulkan for AMD GPUs).
2. Backend selection — Vulkan (universal), NVIDIA CUDA, AMD ROCm/HIP, Intel SYCL, or CPU only.
3. Dependencies — the manager detects your package manager (apt/dnf/pacman/zypper) and shows a backend-specific dependency list; Install Missing installs them with a graphical password prompt (pkexec, falling back to sudo).
4. Option A — build from source — clones llama.cpp into ~/llama.cpp, does a clean CMake build for the selected backend (all other backends explicitly disabled), builds llama-server on all cores and symlinks it into ~/.local/bin.
5. Option B — pre-built binary — downloads a matching Linux release asset from llama.cpp’s GitHub Releases and installs it into ~/.local/bin (runtime libraries kept under ~/.local/lib/llamatray/llama.cpp).
6. Live log — every step streams to a copyable log panel with a one-click Copy Log button, useful for GitHub issues.
7. Clean removal — the Remove llama.cpp action deletes ~/llama.cpp and the installed binary after confirmation; it never touches your settings or profiles.

LlamaTray automatically finds llama-server in ~/.local/bin or ~/llama.cpp/build/bin, so no extra configuration is needed afterwards.