Getting Started

Install LlamaTray, launch it, and start your first local llama-server.

What you need

• A Linux desktop environment (Wayland or X11).
• llama-server from llama.cpp — LlamaTray is a manager for it, not a replacement. If you don’t have it yet, the built-in llama.cpp manager (bottom bar of the app) can build or download it for you.
• A GGUF model file — or use the built-in HuggingFace downloader.

Install

Arch Linux / AUR

yay -S llamatray-git
# or: paru -S llamatray-git

Launch from your application menu, or run LlamaTray in a terminal.

Ubuntu / Debian / Fedora

git clone https://github.com/DolbyDAX2/LlamaTray.git
cd LlamaTray
chmod +x install.sh
./install.sh

The installer detects your distribution, creates a Python virtual environment, installs the dependencies, and registers a llamatray launcher plus an application-menu entry. Launch with:

llamatray

Manual

git clone https://github.com/DolbyDAX2/LlamaTray.git
cd LlamaTray

python -m venv venv
source venv/bin/activate
pip install -r requirements.txt

python -m LlamaTray

See the installation guide for what the installer does step by step.

First launch

1. Open LlamaTray from your application menu (or llamatray / LlamaTray in a terminal).
2. The window shows three tabs: Main, Settings and Profiles.
3. If the system tray is available, you can also start/stop the server directly from the tray icon’s menu.

GNOME users: Qt tray icons need the AppIndicator extension (gnome-shell-extension-appindicator). If LlamaTray says the tray is unavailable, see Troubleshooting.

Quick start — Single Model mode

1. Keep Single Model selected (the default) under the Settings tab.
2. On the Main tab, select a local .gguf file — or click Download from HF to search HuggingFace and download one.
3. Configure GPU layers, context size, port, sampler preset, optional mmproj and extra parameters.
4. Review the generated llama-server command in the live preview, then click Start Server.
5. When the server is up, use Open Web UI to open llama.cpp’s web interface in your browser.

Quick start — Router Mode

1. Select Router Mode under the Settings tab.
2. Choose the directory that contains your GGUF files.
3. Configure auto-load, Jinja templates, GPU layers, context size and port.
4. Start the server — the model list refreshes automatically from the router API.
5. Use Load / Unload next to each model (states: Loaded, Unloaded, Loading).

Router Mode requires a llama-server build that supports --models-dir and the model router API.

Where your data lives

These files survive an uninstall — the uninstaller never deletes ~/.llamatray.

Next steps

• Usage — day-to-day workflows and the tray menu
• Configuration — every setting, with defaults
• Troubleshooting — common issues and fixes