Let AI Answer the Phone: 24/7 Voice Assistant Architecture and Real Latency Data

Letting an AI answer your company phone is no longer experimental: an assistant that picks up the call, understands the question and replies in natural speech is built from three layers — the telephony layer, the speech-to-text layer and the language model that produces the answer. This article covers all three separately, making it clear which piece comes from where.

We run such an assistant on our own company line: it answers calls outside business hours and when the lines are busy, handles common questions and hands the matter over to us when needed. The architecture and cost framing below reflects the logic of that setup.

The critical question is this: is AI fast enough for a phone conversation? Three seconds of silence is tolerated in chat, but on a call it makes the other party think the line dropped. Below we answer that with latency measured from our own production traffic, not with estimates.

A Voice Assistant Is Really Three Separate Layers

"Putting AI on the phone" sounds like buying a single product, but technically it is three independent pieces chained together. Separating them clarifies both the cost and where to look when something breaks.

1. Telephony layer (carrying the audio). The infrastructure that gets the call to you: your PBX, your virtual switchboard service or your carrier's call routing. Digitising the audio, holding, transferring and ending the call belong here. No AI is involved.

2. Speech-to-text and text-to-speech. Turning the caller's sentence into text (speech recognition) and the assistant's reply into audio (speech synthesis). In real-time phone calls, speech recognition is usually handled inside the telephony platform's own pipeline, because that is where the live audio stream is accessible.

3. The brain layer: the language model. The part that understands the transcribed question, combines it with your company knowledge and writes the answer. This is what makes the assistant feel intelligent, and the Onysoft API provides exactly this layer: one OpenAI-compatible endpoint, a choice of 739+ models, billing in Turkish Lira.

The practical consequence: you do not need to replace your telephony infrastructure to build a voice assistant. You add an API call to your existing switchboard flow, and the brain layer comes from outside.

Latency on the Phone: Real Measurements

The acceptable silence window on a call is narrow. People find roughly a one-second pause natural in conversation; beyond two seconds the "did we get disconnected?" doubt begins. So the brain layer's budget is typically around 1.5 seconds.

Measured from our own production traffic over the last 60 days (successful requests only):

Model class	Requests	Average response	Average output
Fast class (flash-lite)	4,553	1,561 ms	53 tokens
Short replies (≤120 tokens), fast class	—	1,471 ms	≤120 tokens
Mid class (mini)	873	2,056 ms	189 tokens
Top class (pro/reasoning)	2,954	8,851 ms	183 tokens

Three conclusions follow. First: fast-class models are suitable for the phone — a 1.5-second average brain latency, added to recognition and synthesis time, still leaves a conversational flow. Second: top-class reasoning models cannot be used live — an 8.8-second average hits a wall in a real call; save those for post-call work such as summaries. Third: output length is latency. If the model writes 400 tokens, the caller waits for all of them. Putting "answer in at most two sentences" in the system prompt gains more than switching models.

Measured 7 August 2026, last 60 days, api.onysoft.com production traffic. Response time is end-to-end API time; telephony and audio conversion latency are not included.

Speech Synthesis: Turning the Answer Into Audio

The language model produces text; what the caller hears is that text spoken. Speech synthesis models are available in the Onysoft catalogue and are called through the /v1/audio/generate endpoint.

curl https://api.onysoft.com/v1/audio/generate \
  -H "Authorization: Bearer sk-ony-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"elevenlabs/text-to-speech-turbo-2-5","prompt":"Hello, how can I help you?"}'

One technical limit up front: this endpoint is asynchronous. The request returns a task_id and you poll /v1/audio/status/{task_id} for the result. That model fits scenarios where the audio is produced ahead of time and stored — greeting announcements, ready answers to frequent questions, campaign messages, voice notifications.

For sentence-by-sentence synthesis in the middle of a live call, you use your telephony platform's own real-time voice engine. In practice this is resolved as follows: frequently repeated answers are synthesised once and cached, and only person-specific sentences are generated live. Most of what an assistant says is repeated anyway, so this approach cuts both latency and cost.

What Does It Cost? An Open Calculation

Take a concrete scenario: 100 calls per day, an average of 6 conversational turns per call (caller asks, assistant answers). Assume roughly 800 tokens of context per turn and roughly 60 tokens of reply.

Daily token volume: 100 × 6 × 800 = 480,000 input, 100 × 6 × 60 = 36,000 output tokens.

Item	Calculation	Daily	Monthly (30 days)
Input (fast class, $0.45/1M)	0.48M × 0.45	$0.216	$6.48
Output (fast class, $3.75/1M)	0.036M × 3.75	$0.135	$4.05
Brain layer total		$0.35	$10.53

So roughly ten dollars a month in language model cost covers 100 calls a day. Add your telephony provider's own fee and, if used, speech synthesis — and if you cache your standard answers, synthesis scales with the number of distinct sentences rather than the number of calls.

The real cost driver is not the call count but the length of context you carry into every turn. Each 500 tokens you add to the system prompt becomes 300,000 tokens across 600 daily turns. Retrieving only the relevant piece of company knowledge per question, instead of embedding everything in the prompt, easily halves this line.

Prices are sale prices and reflect catalogue values at the time of writing; see the model list and cost calculator for current figures.

What Makes an Assistant Useful Is Its Limits, Not Its Model

Most voice assistant projects fail on behaviour design, not model choice. Four rules make AI useful on the phone:

Keep it short. Put a length limit in the system prompt. A five-item list reads fine on screen; on a call the same list loses the listener. "At most two sentences, ask one question if needed" is both natural and fast.

Let it admit what it does not know. A language model tends to fill gaps; an invented price or wrong opening hour on the phone is a direct loss of trust. "If unsure, do not guess — hand the matter to the team" is the assistant's most valuable trait.

Keep the handover path open. The goal is not to finish every call but to finish what it can and pass the rest on with a clean summary. Define the handover threshold: anything unresolved after two turns, a complaint tone, payment or cancellation requests.

Do not hide that it is AI. Saying so in the greeting is both honest and good expectation management; callers who know they are talking to an assistant speak more clearly, and the experience improves.

Compliance Points to Watch

A phone call counts as personal data processing by virtue of being a recording, even when no personal details are spoken. Three headings matter when building a voice assistant.

Notice and disclosure. The presence of recording and of an automated system should be stated at the start of the call. If you already have a call recording announcement, adding the assistant notice there is the most practical route.

Data minimisation. The text sent to the model should not contain identity details that are unnecessary for the task. Caller number, national ID or card data should not enter the prompt, and should be masked where required. This is correct for both compliance and security.

Retention period. Define a retention period for recordings, transcripts and model responses — and actually delete when it expires. This is the most commonly skipped item in voice assistant deployments: transcripts get treated as "logs" and kept indefinitely.

On the Onysoft side, requests are billed from Türkiye with corporate e-invoicing; for details of our data processing approach see KVKK-compliant AI usage.

This section is general information, not legal advice.

Where to Start

Rather than building the whole voice assistant at once, starting from a low-risk corner works better.

Start with out-of-hours. Calls arriving when nobody answers are the lowest-risk ground: the assistant's alternative is not a flawless human but a phone that rings unanswered. Every call handled here is a net gain.

Test the brain layer in text first. Before going near a voice setup, try the assistant's system prompt as text in the Playground. Write out your 20 most common questions and read the answers; sentences that will be heard on the phone must be right on screen first.

Let measurement drive model choice. Start with a fast-class model; if answer quality falls short, move one class up and measure latency again. If you are unsure which model fits, OnyRouter classifies the question and routes it to a suitable model.

Track the handover rate. How many calls the assistant closes itself versus passes on is the only meaningful success metric. If that ratio does not improve over time, the problem is not the model but the knowledge you gave the assistant.

Setting up the brain layer takes minutes: create a free account, get your API key and make your first call with the example in the documentation.