Watch a slick AI-voice demo and the machine answers the instant you stop talking. Get on a real call and there's a beat of silence first — sometimes barely noticeable, sometimes just long enough to make you say "hello?" again. That gap is the single most honest signal of how hard voice AI actually is.
Where the pause comes from
The demo felt instant because it skipped the phone network and ran everything on one machine. A real call can't. Between you finishing a sentence and hearing a reply, a live call has to travel a chain — and every link costs time:
Your voice travels down the line, gets transcribed into text, the model reads it and decides what to say, that reply is turned back into speech, and the audio travels back to you. Add it up and a good system lands somewhere in the region of one to two-and-a-half seconds — on a bad network, worse. The demo hid four of those five steps.
Why the pause is really a decision, not just a delay
Here's the part that surprises people: a big chunk of that "latency" isn't processing time at all — it's the system waiting to be sure you've finished.
People signal the end of a turn with tone, pace and a breath, all without thinking. Software has to infer it, and it faces a genuine dilemma every single time you pause:
- Answer the moment you go quiet, and it will cut you off the instant you take a breath mid-sentence.
- Wait long enough to be safe, and every exchange feels sluggish.
This is "endpointing", and it's the difference between a call that feels like a conversation and one that feels like talking to a kiosk. Tuning it is not about raw speed; it's about judgement — reading the difference between "I've finished" and "I'm thinking."
Why shaving it matters
Below roughly a second of round-trip, a call feels natural; above two, people start to talk over it or repeat themselves, which makes everything worse. So the work isn't glamorous — it's trimming a hundred milliseconds here, overlapping steps that used to run one after another, starting to think before you've completely finished — all while never being the thing that interrupts a caller mid-thought.
When someone tells you their AI answers "instantly", the honest response is: on a real phone line, nothing does. The question is whether the pause is short enough, and smart enough, that you stop noticing it.
🎧 Hear this discussed in our audio briefing — listen here.
Related: How an AI phone call works, end to end · Speech-to-speech, explained