The whole voice stack, behind one gateway you run anywhere
The same engine that powers the hosted platform ships as a gateway you can run on your own infrastructure: a reverse proxy in front of a real-time WebRTC voice pipeline and the agent runtime. It carries no model weights — bring your own LLM key and run it on a single small VM. Need the speech models on-prem too? Add the separate Whissle models service and nothing ever crosses your network boundary.
One entrypoint, everything else pluggable
Frontends — browser, phone, macOS, embed — speak to the gateway on :9000 over HTTPS and WebRTC. The lightweight container holds just the proxy, the voice pipeline, and the agent runtime. Speech models, the LLM, and the database are all things you point it at.
Separate ASR/TTS/video engines, optional — wired via MODELS_HOST. Run it on-prem for private speech.
Deepgram · Sarvam · Gemini · Claude · Simli — the default when there's no models service.
External and stateless in prod; a convenience in-container DB for single-box dev.
Two ways to run it
The backend is always the same small, stateless container. What changes is where the speech models live: on a provider's API, or on your own hardware.
Backend on external APIs
defaultThe gateway backend ships no model weights — STT/TTS/LLM/avatar run over provider APIs. That's the whole footprint: a stateless container on a small VM.
- Footprint
- ~500 MB RSS
- Fits
- an e2-small (2 GB)
- Speech models
- provider APIs
- Database
- external, stateless
Backend + your own models
MODELS_HOSTRun the separate Whissle models service — ASR / TTS / video — on your own box or GPU and point the backend at it. Audio and transcripts never leave your network.
- Backend
- still ~500 MB
- Models service
- ~5 GB, on your GPU
- Speech models
- local, on-prem
- Wiring
- one env var: MODELS_HOST
Every stage is a real component
A turn passes through parts you can open, tune, redact, and swap — not a sealed black box. Change the provider, retune the endpointer, gate the noise.
Speech in
Streaming STT with endpointing, a noise gate, and an addressee gate — Deepgram / Sarvam by API, or the on-prem Whissle ASR.
Reasoning + tools
The agent runtime: your prompt, knowledge base, and real tools — bookings, records, SMS, webhooks — over Gemini or Claude.
Speech out
TTS with emotion and mid-call language switching — Deepgram Aura, Sarvam, Cartesia, ElevenLabs, or local Kokoro.
Avatar (optional)
A lip-synced video presence via Simli, or client-side rendering — wire it in per agent, or leave it off.
Private by default
Add the on-prem models service and audio, transcripts, and structured metadata never leave your network. Swap the managed database for your own; bring your own keys. A deployment that lives or dies on compliance keeps the boundary.
Yours to tune
Opt into the advanced turn-taking levers — speculative tool pre-warm, a shadow-LLM draft head, live turn-completeness — the same ones we A/B on the hosted platform. Off by default, documented, reversible.