Features

Built for running local models on Linux

Every capability below ships in LlamaTray v1.5.4. Nothing here is a roadmap item — this is what the application does today.

Server management

Start, stop, inspect — without a terminal

LlamaTray manages the llama-server process for you: it finds the binary, checks the port, builds the command and keeps a live log of every step.

One-click start & stop

Launch llama-server from the Main tab or straight from the tray menu. Stopping the server — or closing LlamaTray — terminates the process cleanly, with an HTTP shutdown request first and process/port cleanup as a fallback.

Live command preview

Before you start, LlamaTray shows the exact llama-server command it will run. What you see is what executes — no hidden flags.

Web UI shortcut

A one-click button opens the llama.cpp web interface in your default browser once the server is up on http://127.0.0.1:<port>.

Live log panel

Every step — binary lookup, port check, startup, output, shutdown — is streamed to a log panel you can copy for bug reports.

Run modes

Single Model and Router Mode

Single Model

Select a local .gguf file or download one from HuggingFace.
Configure GPU layers, context size, port, sampler preset and extra parameters.
Attach an --mmproj multimodal projector for compatible vision-language models.

single model

llama-server -m ~/models/model.gguf --n-gpu-layers 99 --ctx-size 32768 --port 8080

Router Mode

Point LlamaTray at a directory containing GGUF files and serve them through llama-server's model router.
Model states (Loaded, Unloaded, Loading) are read live from the GET /models endpoint.
Load or unload any model with a button next to it.

router mode

llama-server --models-dir ~/models --no-models-autoload --jinja --host 127.0.0.1 --port 8080 -ngl 99 --ctx-size 32768

Router Mode requires a llama-server build that supports --models-dir and the model router API.

Monitoring

Watch your resources while the model runs

The monitor widget refreshes every second and degrades gracefully: whichever backend is available on your system is used, and missing metrics simply stay hidden.

Metric	Source
CPU & RAM	psutil
NVIDIA GPU / VRAM	NVML (nvidia-ml-py) → nvidia-smi
AMD GPU / VRAM	rocm-smi → amdgpu sysfs

Honest about your hardware

GPU metrics are only shown when the corresponding tooling is installed. On systems without GPU monitoring support, LlamaTray keeps working and simply reports CPU and RAM.

Profiles & configuration

Your setup, remembered

Named profiles cover both Single Model and Router configurations. Profiles are stored in ~/.llamatray/profiles.json, application preferences in ~/.llamatray/config.json — both restored automatically between sessions.

Setting	Description	Default
GPU Layers	Layers offloaded to the GPU	99
Context Size	512 – 1,000,000	32768
Port	1024 – 65535, bound to 127.0.0.1	8080
Sampler Preset	Neutral, balanced, creative or precise	Custom
Extra Parameters	Any additional llama-server flags	optional
mmproj File	Multimodal projector (Single Model mode)	optional

Sampler presets

Presets map directly to llama-server sampling flags — pick one, or switch to Custom and edit freely.

Preset	Flags
Neutral	--temp 0.7 --top-p 0.9 --top-k 40 --min-p 0.0 --repeat-penalty 1.0
Balanced	--temp 0.5 --top-p 0.95 --top-k 20 --min-p 0.0 --repeat-penalty 1.05
Creative	--temp 1.0 --top-p 0.99 --top-k 100 --min-p 0.0 --repeat-penalty 1.0
Precise	--temp 0.1 --top-p 0.5 --top-k 10 --min-p 0.0 --repeat-penalty 1.1

HuggingFace downloader

Search GGUF repositories without leaving the app.
File names and exact sizes are fetched through the HuggingFace Tree API before you download.
In Router Mode, downloads default to your configured models directory.
Cancelled or failed downloads remove their partial file automatically.

How it works →

llama.cpp manager

Hardware pre-flight scan using nvidia-smi, rocminfo, vulkaninfo, clinfo and lspci.
Backends: Vulkan, NVIDIA CUDA, AMD ROCm/HIP, Intel SYCL or CPU only — each with its own dependency list for apt/dnf/pacman/zypper.
Compile from source into ~/llama.cpp, or install a pre-built release binary if you have no build tools.
Live, copyable build log and a clean-removal option for switching versions.

How it works →

LlamaTray finds llama-server automatically in ~/.local/bin or ~/llama.cpp/build/bin — no extra configuration needed.

System integration

A native Linux citizen

Tray-first, window optional

Show/hide the window, start or stop the server and quit — all from the system tray menu. Minimizing to the tray is a saved preference.

Wayland & desktop support

Native Wayland and KDE Plasma support. On GNOME, the AppIndicator extension provides the tray; when no tray is available LlamaTray safely continues in window mode.

Localized interface

The interface ships with English and Turkish translations, switchable in-app.