Troy enables fine-tuning of Large Language Models (LLMs) directly on your Mac, leveraging Apple Silicon and the MLX framework. This tool simplifies the complex process of LLM training into a single YAML configuration file and a straightforward command. Troy eliminates the need for CUDA or cloud infrastructure, allowing models to train efficiently on your local machine.

• Simplified Workflow: Train LLMs with just one YAML file and one command.

• Apple Silicon Optimized: Built on MLX for maximum performance on M1 chips and later.

• No Cloud Required: Train models locally, saving costs and enhancing privacy.

• Comprehensive Training Options: Supports SFT, DPO, and ORPO fine-tuning tasks.

• Automated Configuration: Infers batch size from available memory and auto-detects data formats (Alpaca, ShareGPT, etc.).

• Dataset Generation: Optionally generate datasets using a local teacher model or a mesh network of iPhones.

• Cross-Platform Export: Export trained adapters for use with llama.cpp, Ollama, LM Studio, or even directly on iPhones via Swift applications.

• Unified Memory Utilization: Takes full advantage of Mac's unified memory for training larger models than typically possible on consumer hardware.

• Efficient DPO: Implements a memory-efficient DPO training method by recovering the reference model from the policy model.

• Integrated Chat: Interact with your newly trained adapter directly within the terminal.

Troy is open-source under the Apache-2.0 license, requiring an M1 chip or later. It provides commands like troy doctor to check system compatibility, troy init to set up configuration, troy data synth for dataset creation, troy train for the core fine-tuning process, troy chat for interaction, and troy export for deployment.