Founder of Soup, an open-source LLM fine-tuning CLI. Made 8B models trainable on a 4 GB laptop GPU. 18, Astana, KZ

github

Alpamys Makazhan • Peerlist

<h3>Soup CLI: Streamlined LLM Fine-Tuning on Consumer Hardware</h3>Soup CLI is a powerful command-line interface designed to simplify and optimize the process of fine-tuning large language models (LLMs), even on hardware with limited resources. It introduces innovative techniques like layer streaming and NF4 quantization to enable the fine-tuning of models like Llama-3.1-8B on a 4 GB laptop GPU.

<h3>Key Features:</h3><ul><li>Layer Streaming: Load and process LLM layers one by one from CPU RAM or NVMe storage, drastically reducing VRAM requirements. Peak VRAM is bounded by a single layer, not the entire model.</li><li>NF4 Quantization: Further reduces memory footprint by quantizing streamed layers to 4-bit precision, making larger models accessible on consumer GPUs.</li><li>Efficient Preference Tuning: Supports DPO, ORPO, SimPO, and KTO fine-tuning methods over streamed bases, with DPO's reference model efficiently managed without duplicating weights.</li><li>Broad Model Support: Compatible with numerous architectures including Llama, Qwen, Mistral, Gemma, and Phi families.</li><li>Comprehensive Fine-tuning Options: Offers a wide range of training methods including SFT, Pretrain, GRPO, and more, with extensive PEFT support (LoRA, QLoRA, DoRA, VeRA, PiSSA, ReLoRA).</li><li>Built-in Quality Gates: Integrates features like soup ship for automated SHIP/DON'T-SHIP verdicts based on task performance and catastrophic forgetting, and soup diagnose for post-training failure mode analysis.</li><li>Compliance and Security: Includes tools for generating compliance documentation (HIPAA, SOC 2, EU AI Act) and preventing supply-chain attacks on adapters.</li><li>Cross-Platform Compatibility: Supports Linux, macOS, and Windows, with specific optimizations for Apple Silicon via an MLX backend.</li><li>Reproducibility: Employs mechanisms like soup lock and soup env lock to ensure reproducible training runs by pinning base models, datasets, and environments.</li></ul>Soup CLI aims to democratize LLM fine-tuning, making advanced techniques accessible to a wider audience by optimizing for resource-constrained environments without compromising on model quality or correctness. Every measurement and potential defect is published, ensuring transparency in the development process.

Soup CLI - fine-tune an 8B LLM on a 4 GB GPU

Leading positioning, go-to-market and fundraising for an open-source LLM fine-tuning platform. Grew to 60,000+ downloads organically, with zero marketing budget. Positioning the product around reliability and compliance for regulated industries (banking, healthcare, government).

Co-founder & CEO — Soup (trysoup.dev)

Secure Docker sandbox for AI coding agents. Lets you run Claude Code at full speed without exposing secrets or the host machine: secret isolation, command policy enforcement, network controls, a full audit trail of everything the agent did, and one-command rollback. Written in Go. github.com/MakazhanAlpamys/claudeshield

ClaudeShield

Analytics platform over 210,000+ Kaspi.kz marketplace products — ingestion pipeline, price and demand modelling, and an ML forecasting dashboard. Next.js 16, React 19, FastAPI, scikit-learn, Tailwind 4.

Kaspi Analytics Platform

Ensemble ML system (LightGBM, RandomForest, ExtraTrees, AdaBoost, XGBoost and stacking) identifying exoplanet candidates in NASA Kepler, K2 and TESS mission data, with SHAP explainability, probability calibration, adjustable decision thresholds and independent TOI/K2 validation. NASA Space Apps Challenge — winner of the Astana round and Global Nominee. Challenge: "A World Away: Hunting for Exoplanets with AI".

NASA Exoplanet Detection AI

RAG-powered urban assistant recommending places and events in Astana from the user’s location, preferences and context. React 19, Node.js/Express, Gemini 2.0, TypeScript. 1st place, Google Developer Groups Astana 2025.

Smart City Advisor

FairTender is a machine learning platform designed to analyze and detect suspicious government procurement transactions. Using XGBoost with SHAP explainability, the system provides interpretable results and highlights risk factors. The solution includes an interactive Streamlit web app for data upload, visualization, and regional analysis.

FairTender

AiBio is an innovative AI-driven platform for biology education. It provides interactive 3D models, a lesson builder for teachers, and an AI tutor for personalized learning. Developed with Node.js, React, PostgreSQL, and Gemini AI, the project won 1st place at the PowerUp Hackathon.

AiBio

Chess.AI is an intelligent chess learning platform that combines AI-powered gameplay with interactive learning features. The system integrates Stockfish for engine analysis and Chess.js for move validation, offering adjustable difficulty levels, video tutorials, and teacher–student collaboration tools. Technologies: React, Node.js, PostgreSQL.

Chess.AI

FakeDetectorNews is an AI-driven web application that verifies the credibility of online news articles. The system integrates Gemini LLM to analyze content, generate reliability scores, and provide fact-checking insights. Built with React, Node.js, and PostgreSQL, the platform delivers a user-friendly interface and scalable backend architecture.

FakeDetectorNews

Fine-tune and post-train any open LLM in one command. 20 training methods (SFT, DPO, GRPO, PPO, KTO, ORPO, distillation, reward models), 142 model recipes, 9 quantization formats, multi-GPU via DeepSpeed ZeRO and FSDP2, and one-command export to GGUF, Ollama and HuggingFace. Two capabilities nothing else has: layer streaming, which streams the frozen base model from CPU RAM one decoder layer at a time so Llama-3.1-8B fine-tunes in 3.32 GB on a 4 GB laptop GPU — published as a preprint with a DOI (10.5281/zenodo.21771064); and a full training-provenance stack (CycloneDX ML-BOM, SLSA-3 attestation, EU AI Act Annex XI/XII docs, signed adapters, backdoor scanning). 153,000 lines of Python, 16,735 automated tests, 167 releases in five months, 78,000 downloads. Apache-2.0. trysoup.dev · github.com/MakazhanAlpamys/Soup

Soup — Open-Source LLM Fine-Tuning CLI

AI assistant for Dubai building-code compliance. Hybrid RAG pipeline over five Dubai Municipality regulatory documents, so architects and consultants stop losing hours to overlapping PDFs. Next.js 15, TypeScript, LangChain, Gemini 2.5 Flash, Supabase pgvector, Zod. 1,202 passing tests.