llama.cpp Manager

The built-in manager that detects your hardware, installs llama.cpp dependencies, and gives you two ways to get a llama-server binary: compile from source (Option A) or download the pre-built release (Option B).

LlamaTray ships with a dedicated llama.cpp Manager so you don’t have to build or install llama.cpp manually. It scans your hardware, checks (and can install) the distribution packages you need, and then installs llama-server for you — either compiled from source or as a ready-made binary.

Opening the manager

Click the 🛠 llama.cpp Manager button in the bottom toolbar of the main window. The manager opens as a dialog with four areas:

• Hardware Detection — what GPU you have and which backend is recommended
• Dependencies & Build Tools — package status for your distribution
• Compile from Source (Option A) — build llama.cpp for your hardware
• Download Pre-built Binary (Option B) — fetch the newest published release

A log panel at the bottom shows every step, with a progress bar and a 📋 Copy Log button.

Hardware detection

When the dialog opens it scans your system without blocking the UI. It checks for the tools nvidia-smi, nvcc, rocminfo, vulkaninfo, clinfo, lspci, hipcc and icpx, lists NVIDIA GPUs with nvidia-smi -L, identifies the GPU vendor through lspci, and probes the AMD ROCm stack with rocminfo --devices.

Based on what it finds, the manager prints a recommendation banner:

The recommendation is a starting point — you can always pick any backend manually.

Choosing a backend

The Target Backend group offers five options:

When you compile, the selected backend is turned ON and all other GPU backends are explicitly set to OFF, so a single build targets exactly one accelerator.

Dependencies & build tools

The Dependencies & Build Tools panel shows:

• your detected package manager (apt, dnf, pacman or zypper),
• the build tools it found (cmake, git, g++/clang, make),
• the packages required for your distribution and selected backend.

Check verifies that each package is actually installed — through pacman -Q, dpkg-query -W or rpm -q, not just by looking for binaries. Missing packages are listed in the panel and Install Missing (sudo) installs them with graphical authorization (pkexec), falling back to sudo when pkexec is unavailable.

Examples of what gets installed per backend (your distribution’s package set is shown in the dialog):

# Base build tools (pacman)
base-devel cmake git

# Vulkan extras (pacman)
vulkan-headers vulkan-icd-loader shaderc spirv-headers

# CUDA extras (apt)
nvidia-cuda-toolkit nvidia-cuda-dev libcuda1

A few backends need extra attention on some distributions:

• AMD ROCm on Ubuntu/Debian — the ROCm packages are not in the standard repositories. You must first add AMD’s official APT repository (repo.radeon.com); the manager will tell you when this is the case instead of attempting a broken install.
• NVIDIA CUDA on Ubuntu/Debian — the packages require the official NVIDIA drivers to be present. The manager warns you if the driver appears to be missing and suggests Option B as an alternative.
• Intel SYCL — the icpx compiler cannot be installed with standard package managers; you install the Intel oneAPI SDK manually (or use Option B).

Option A: Compile from source

Compile from Source (Option A) builds llama.cpp specifically for the selected backend and local system. It:

1. clones llama.cpp into ~/llama.cpp,
2. runs CMake with your backend flag,
3. compiles the project (this can take a few minutes),
4. installs the resulting llama-server to ~/.local/bin/llama-server.

Use Start Build to begin and Stop to cancel at any time; every step is written to the log with a progress bar. An old CMake build cache in ~/llama.cpp/build is cleared before configuring.

Option B: Download the pre-built binary

Download Pre-built Binary (Option B) skips compilation entirely. It queries the latest published llama.cpp GitHub release (pre-releases included), picks the Linux x64 asset — preferring a Vulkan build when several are available — and downloads it with a progress bar.

After the download, the archive is extracted and installed into ~/.local/lib/llamatray/llama.cpp together with the shared runtime libraries (.so files). A small launcher script is written to ~/.local/bin/llama-server that sets LD_LIBRARY_PATH to those libraries and execs the real binary.

Where the files live

LlamaTray doesn’t care which option you used: when it starts a server it first looks for llama-server on your PATH, then falls back to well-known locations (/usr/bin, /usr/local/bin, ~/.local/bin, ~/llama.cpp/build/bin, ~/llama.cpp/server). The log shows the exact match, e.g. ✓ llama-server found: ~/.local/bin/llama-server.

The release tag of what you installed (a GitHub tag such as b10909, not a commit hash) is shown next to each option and remembered separately for Option A and Option B across sessions.

Removing the installation

The 🗑 Remove llama.cpp button cleans up everything the manager installed:

~/llama.cpp                          # sources + build tree (Option A)
~/.local/lib/llamatray/llama.cpp    # pre-built runtime (Option B)
~/.local/bin/llama-server           # binary / launcher
~/.llamatray/llamacpp_install.json  # manager metadata

The dialog lists exactly which paths will be deleted and asks for confirmation. Your LlamaTray settings, profiles and the rest of ~/.llamatray are preserved. Cleanup is blocked while a build or download is still running, and while a llama-server is active — stop the server first.