Fluencto
A voice receptionist for clinics that understands patients, checks schedules and services through tool calling, and responds with natural audio — all within one WebSocket session.
The challenge
In a voice conversation, a correct answer that arrives late still feels wrong. The system must coordinate transcription, reasoning, database access, and synthesis without blocking the flow — while keeping history consistent when the user interrupts.
Real-time architecture
- End-to-end streaming. The browser sends PCM16 through AudioWorklet; Deepgram, Claude, and ElevenLabs stream their work instead of waiting for full outputs.
- Context-aware tools. The agent queries services, professionals, schedules, and FAQs in SQLite through tool calling.
- Consistent barge-in. New speech cancels the LLM and TTS, clears the player, and preserves a valid conversation history.
- Observable latency. Every stage is measured against the audio clock, separating endpointing, first token, first sentence, and first sound.
Current status: the median measured before the latest fix was approximately 3.2 s. Pre-connection has been implemented, but it still needs notebook validation and repository sync before the public demo.
