This release introduces a redesigned tabbed interface and comprehensive llama-server Router Mode support. Users can now manage multiple GGUF models from a single models directory, load or unload models dynamically, monitor their current states, and download new models directly into the router directory through the optimized HuggingFace downloader.
What’s New:
- Tabbed Interface: The application interface is now organized into three tabs — Main, Settings, and Profiles. Server controls and system monitoring are available on the Main tab, advanced configuration options are located under Settings, and profile management has its own dedicated tab.
- Router Mode: A new operating mode selector allows users to switch between Single Model and Router Mode. Router Mode launches
llama-serverwith a models directory instead of a single model file and provides options for automatic model loading, Jinja templates, GPU layers, context size, port, and additional parameters. - Dynamic Model Management: Models exposed by the router are retrieved through the
llama-serverAPI and displayed with readable states such as Loaded, Unloaded, Loading, and Unknown. Models can be loaded or unloaded directly from LlamaTray without restarting the server. - Router Profile Support: The existing profile system now supports both Single Model and Router configurations. Router mode, models directory, auto-load preference, Jinja setting, GPU layers, context size, port, and additional parameters can be saved and restored through profiles.
- Improved HuggingFace Downloader: Repository files and their exact sizes are now retrieved through the HuggingFace Tree API, significantly reducing file listing times. A fallback mechanism is retained for compatibility when the Tree API is unavailable.
- HuggingFace Router Integration: The “Download from HF” button remains available in Router Mode. The configured models directory is automatically selected as the download destination, and the router model list is refreshed after a download completes while the server is running.
- Collapsible Settings Sections: The operating mode selector, Advanced Settings, and Launch Command Preview sections can now be collapsed by clicking their headers.
- Centralized Version Management: Application version information is now maintained from a single source and shared by the main window, About dialog, package exports, and installation script.