About HomeBred-LLM

HomeBred-LLM is a cross-platform desktop application designed to run and manage Large Language Models (LLMs) entirely on your local hardware. It eliminates the need for external services like Ollama or Docker, offering a self-contained solution for interacting with open-weight AI models offline. The application is built with Avalonia UI and leverages llama.cpp via LLamaSharp for efficient, in-process inference.

Key Features:

• Model Management: Download LLMs directly from HuggingFace, with a progress bar for real-time updates. Browse models by quantization and size.

• Configuration: Fine-tune model behavior with sliders for temperature, top-p, top-k, context size, GPU layers, and more.

• LoRA Adapters: Apply pre-trained LoRA adapters to models for specialization, with adjustable blend scales and toggle functionality.

• Performance Monitoring: Real-time analytics including GPU utilization, VRAM usage, temperature, CPU and RAM usage, tokens per second, and time-to-first-token.

• Local API: An optional, opt-in API endpoint (OpenAI-compatible) for chat completions and embeddings, bound to localhost.

• Cross-Platform: Runs on Windows, Linux, and macOS.

• No Cloud, No Telemetry: Ensures privacy and offline functionality.

The application stores models, configurations, and analytics data locally in a SQLite database, ensuring all operations are performed on your hardware without any cloud dependency.