View Project
SmolChat is an open-source, native-Android application helping users run large-language models on their mobile devices offline and interact with them in a ChatGPT-like interface.
- The models and the chats remain completely offline, on-device, ensuring data privacy and no network reliance.
- Download any model available in the GGUF format and start using it right away. You can customize the system prompt and inference parameters (temperature, min-p) for controlling the text generation.
- Create pre-defined 'Tasks' which help you perform tasks like text summarization and rewriting quickly, without configuring the chats every time.
SmolChat is an open-source mobile application, licensed under Apache 2.0.
Technical Details
The app uses its own small JNI binding written over llama.cpp, which is responsible for loading and executing GGUF models. Chat, message and model metadata are stored in a local ObjectBox database. The codebase is written in Kotlin/Compose and follows modern Android development practices.
The JNI binding is inspired from the simple-chat example in llama.cpp.
Demo Video:
Interacting with a SmolLM2 360M model for simple question-answering with flight-mode enabled (no connectivity)
Adding a new model, Qwen2.5 Coder 0.5B and asking it a simple programming question
Using a prebuilt task to rewrite the given passage in a professional tone, using SmolLM2 1.7B model
Project (with an APK built): https://github.com/shubham0204/SmolChat-Android