Soup CLI: Streamlined LLM Fine-Tuning on Consumer Hardware

Soup CLI is a powerful command-line interface designed to simplify and optimize the process of fine-tuning large language models (LLMs), even on hardware with limited resources. It introduces innovative techniques like layer streaming and NF4 quantization to enable the fine-tuning of models like Llama-3.1-8B on a 4 GB laptop GPU.

Key Features:

• Layer Streaming: Load and process LLM layers one by one from CPU RAM or NVMe storage, drastically reducing VRAM requirements. Peak VRAM is bounded by a single layer, not the entire model.

• NF4 Quantization: Further reduces memory footprint by quantizing streamed layers to 4-bit precision, making larger models accessible on consumer GPUs.

• Efficient Preference Tuning: Supports DPO, ORPO, SimPO, and KTO fine-tuning methods over streamed bases, with DPO's reference model efficiently managed without duplicating weights.

• Broad Model Support: Compatible with numerous architectures including Llama, Qwen, Mistral, Gemma, and Phi families.

• Comprehensive Fine-tuning Options: Offers a wide range of training methods including SFT, Pretrain, GRPO, and more, with extensive PEFT support (LoRA, QLoRA, DoRA, VeRA, PiSSA, ReLoRA).

• Built-in Quality Gates: Integrates features like soup ship for automated SHIP/DON'T-SHIP verdicts based on task performance and catastrophic forgetting, and soup diagnose for post-training failure mode analysis.

• Compliance and Security: Includes tools for generating compliance documentation (HIPAA, SOC 2, EU AI Act) and preventing supply-chain attacks on adapters.

• Cross-Platform Compatibility: Supports Linux, macOS, and Windows, with specific optimizations for Apple Silicon via an MLX backend.

• Reproducibility: Employs mechanisms like soup lock and soup env lock to ensure reproducible training runs by pinning base models, datasets, and environments.

Soup CLI aims to democratize LLM fine-tuning, making advanced techniques accessible to a wider audience by optimizing for resource-constrained environments without compromising on model quality or correctness. Every measurement and potential defect is published, ensuring transparency in the development process.