Barge-in and turn-taking: letting callers interrupt an AI agent

A caller to a clinic in Jaipur says, "I wanted to ask about, um, the Saturday timing." The agent hears the pause after "um" and jumps in with the wrong answer. Ten seconds later the caller tries to correct it and the agent keeps reading its paragraph. Neither problem is about the AI being unintelligent. It is about knowing when it is whose turn.

How an agent knows the caller has finished

A phone line gives the agent only sound. It cannot see a face or a breath. It has to infer "finished" from silence and, in better systems, from the words themselves: a sentence that ends with a question is likely complete; one that trails off after "and" or "um" is not.

The simplest approach is a silence timer: if the caller is quiet for a set time, the turn is over. Smarter approaches combine the timer with a model that judges whether the sentence sounds complete. Either way there is a trade-off. Wait longer and you rarely cut people off, but every reply is slower (see why the pause matters). Wait less and replies feel quick, but people who pause to think get interrupted.

How interruption works

Barge-in means the caller can speak while the agent is talking, and the agent stops and listens. Humans do this constantly: "yes, yes, I know, just tell me the time."

Technically the agent keeps listening even while it speaks. When it detects the caller's voice, it does three things: stops playing audio at once, discards the part of the reply it has not yet said, and treats the caller's words as the next input. A good implementation also remembers what was actually said before the cut-off, so it does not repeat itself or claim to have said more than the caller heard.

Cora Call lets callers interrupt, and turn-taking is tunable per agent.

Why it sometimes stops for no reason

A false trigger is the agent stopping when the caller did not mean to take the floor. Common causes:

• Background noise. A horn, a pressure cooker, a TV, someone else in the room talking.
• Backchannels. "Hmm", "haan", "okay", "right". They are the caller saying "I'm listening", not "stop".
• Echo. On a speakerphone, the agent's own voice comes back through the caller's microphone and sounds like an interruption. The agent then cuts itself off.
• Coughs and breaths. Short sounds that are not speech.

The opposite fault is real too: an agent that ignores real interruptions because its sensitivity is set too high, so it keeps talking while the caller says "no, wait".

The trade-offs you tune

There is no universal right setting. A quiet reception line, a car-mounted phone and a shop floor need different values. Elderly callers and people dictating numbers need more patience. Start with defaults, read the transcripts, and look for the two signs: the agent answering a half-finished sentence, or the caller saying "hello?" because it did not respond.

Different callers, different patience

A single setting rarely suits everyone. A college student asking about a hostel fee speaks quickly and expects a quick reply. A shopkeeper in a market stall is shouting over traffic. An elderly patient dictating a date of birth pauses between every number. A caller who is unsure of the answer says "uh" often and wants time. Because the agent cannot see any of this, it has to be told how to behave.

Some practical habits help. For number or spelling input, instruct the agent to wait and read the details back for confirmation rather than answer after the first pause. For noisy lines, raise the interruption threshold so ambient sound does not stop the agent. For long explanations, keep each reply short so there are natural places for the caller to jump in, instead of one long block that must be interrupted to be stopped.

Prompt wording matters as well. An agent that is told to "always finish your explanation" will sound deaf, while one told to "stop and respond if the caller speaks" will sound attentive. Check that the prompt and the turn-taking settings are not working against each other.

Finally, remember that the caller's phone, carrier and speakerphone habits are outside your control. If echo is the main problem, a short line asking the caller to use the handset or earphones, used only when you detect it, costs less than a call that keeps interrupting itself.

What a bad turn looks like in a transcript

Turn-taking faults leave clear marks if you know where to look. An agent that jumps in too early produces transcripts where the caller's sentences end in the middle, and the next line starts with the agent answering a question the caller never finished asking. An agent that waits too long produces callers who say "hello?" or repeat the whole question. An agent that is too easy to interrupt produces its own sentences cut to two or three words, again and again, with no caller speech recorded between them: that pattern usually means noise or echo, not a talkative caller.

Read a sample of calls with these three patterns in mind and note which one dominates. Fix that one first. Changing several settings at once makes it impossible to tell which change helped, and a fix for one pattern often causes the opposite one.

Fillers while a tool runs

When the agent needs a tool, such as checking a calendar or looking up an order, the answer can take a second or more. Silence feels like a dropped line. A short filler, "let me check that for you", tells the caller the call is alive. Good practice:

• Keep fillers short and varied; the same phrase every time sounds scripted.
• Say a filler only when the tool is actually slow, not before every lookup.
• Let the caller interrupt a filler, too.
• If the tool fails, say so plainly and offer another way, such as a callback.

Where to start

1. Make a test call and deliberately pause mid-sentence. Does the agent wait?
2. Interrupt it mid-reply. Does it stop within a moment and respond to what you said?
3. Say "hmm" and "okay" while it talks. Does it carry on?
4. Call on speakerphone and from somewhere noisy. Does it cut itself off?
5. Change one setting, then repeat. Review a week of real transcripts before settling.

Hear how it feels in the live demo.