Key Features

• Local Inference: Runs STT (Whisper), LLM (Llama 3.2 1B), and TTS (Piper) entirely on your local machine.

• Smart Turn-Taking: Energy-based VAD (Voice Activity Detection) handles natural conversation flow.

• Barge-In Support: The AI stops talking immediately if the caller interrupts.

• Advanced Memory:

  • Ground Truth: Authority context for factual accuracy.

  • Profile Memory: Remembers names, facts, and preferences across calls.

  • Semantic Context: Retrieves relevant past conversation snippets.

• Telephony: Seamless integration with Twilio Voice via Media Streams.

Tech Stack

• Backend: FastAPI (Python 3.10+)

• STT: faster-whisper

• LLM: Ollama (Llama 3.2 1B)

• TTS: Piper

• Audio: NumPy, Scipy

Performance expectations (CPU)

Stage Latency

Transcription (Whisper) ~600ms
Brain (LLM First Token) ~800ms
Voice (TTS Generation) ~150ms

Total Response Time~1.5s - 2.5s