AI Voice & Telephony
Enterprise AI Voice Call Center
An AI-powered voice call center that answers inbound calls, understands intent, and routes conversations into the right workflow automatically.
Overview
Built an AI-powered voice call center that automatically handles inbound phone calls, identifies caller intent using an LLM, and routes conversations into specialized workflows for different request types. The system conducts natural voice conversations, collects structured information, stores it in a database, and automatically notifies the appropriate team.
Problem
Every inbound call had to be answered, triaged, and routed by a human before any actual work could start. Simple, repetitive requests (status checks, basic intake) consumed the same attention as urgent ones, and nothing was categorized until someone manually wrote it down.
Business Goal
Answer every call immediately, classify intent automatically, and get structured information into the right workflow and the right inbox without a human doing the first pass.
Architecture
System design
A Twilio Voice number feeds live audio into a Power Automate flow, which streams the conversation through GPT for real-time intent detection. Based on the detected intent, the call is routed into one of several specialized conversational flows, each responsible for collecting a specific set of structured fields through natural dialogue rather than rigid IVR menus.
Challenges
What made this hard
TwiML responses need to feel like a conversation, not a phone tree, while still being generated dynamically per turn.
Built a flow that generates TwiML on the fly from the LLM's response text, so the voice prompt changes naturally based on what the caller just said instead of playing from a fixed script.
Misrouted intent early in the call wastes the entire conversation.
Intent detection runs on the first meaningful utterance and is re-checked if the caller's answers don't match the expected shape for that workflow, allowing a mid-call correction instead of a dead end.
Business Impact
What it changed
Reduced manual call handling while ensuring every inquiry is automatically categorized and routed to the correct workflow — no call goes untriaged, and structured data is available immediately instead of after manual note-taking.
Lessons Learned
- Voice UX is a different discipline from chat UX — latency and turn-taking matter more than response quality alone.
- Designing the intent taxonomy up front (before building flows) prevented a lot of rework later.
Future Improvements
- Add sentiment detection to flag frustrated callers for immediate human handoff.
- Persist full call transcripts for QA and intent-model tuning.
Related