🎙 Realtime API — Live Voice Conversations

Low-latency voice assistants over WebSocket. Audio in, audio out — no transcription round-trip.

The Realtime API opens a two-way audio channel between your phone system or web app and the model. The model listens while the user speaks, answers on its own once they stop, and streams the audio back in chunks. In the classic record, transcribe, generate, synthesise chain every step queues behind the last and latency piles up; here everything flows over one connection, so the user hears the start of the answer far sooner.

When to use it

Connecting

GET /v1/realtime?model=openai/gpt-realtime-1.5 HTTP/1.1
Host: api.onysoft.com
Upgrade: websocket
Connection: Upgrade
Authorization: Bearer sk-ony-...

Event Flow

Once connected, everything happens through JSON event messages. A typical turn goes like this:

1. You connect; the server sends session.created as the first message.
2. You send session.update with audio format, voice, turn detection and transcription settings.
3. You base64-encode microphone audio and stream it in chunks with input_audio_buffer.append.
4. The server detects the end of speech (server VAD), emits input_audio_buffer.speech_stopped and starts generating.
5. Answer audio arrives chunk by chunk as response.output_audio.delta events, which you play back.
6. response.done closes the turn and reports the tokens it consumed — billing is based on this.

Session Settings

The first message you send after connecting should be session.update. These are the fields you will use most:

{
  "type": "session.update",
  "session": {
    "type": "realtime",
    "audio": {
      "input": {
        "format":         { "type": "audio/pcmu" },
        "turn_detection": { "type": "server_vad", "create_response": true },
        "transcription":  { "model": "whisper-1", "language": "tr" }
      },
      "output": {
        "format": { "type": "audio/pcmu" },
        "voice":  "marin"
      }
    },
    "instructions": "Türkçe konuş. Kısa ve net yanıt ver.",
    "max_output_tokens": 4096
  }
}

Audio Formats

The wrong format is the most common problem: audio flows but the other side hears noise. Pick the one that matches your line.

Common Events

The full list is long; these are the ones you need day to day.

Example

The smallest working example that connects, sends settings and handles incoming events.

import WebSocket from "ws";

const ws = new WebSocket(
  "wss://api.onysoft.com/v1/realtime?model=openai/gpt-realtime-1.5",
  { headers: { Authorization: "Bearer sk-ony-..." } }   // anahtar SUNUCUDA kalir
);

ws.on("open", () => console.log("bagli"));

ws.on("message", (raw) => {
  const e = JSON.parse(raw);
  switch (e.type) {
    case "session.created":
      ws.send(JSON.stringify(sessionUpdate));   // ilk mesaj
      break;
    case "input_audio_buffer.speech_started":
      stopPlayback();                            // soz kesme
      ws.send(JSON.stringify({ type: "response.cancel" }));
      break;
    case "response.output_audio.delta":
      play(Buffer.from(e.delta, "base64"));
      break;
    case "response.done":
      console.log("token:", e.response.usage);  // faturalama bu
      break;
    case "error":
      console.error(e.error.message);
  }
});

// mikrofon sesini base64 kodlayip akit
ws.send(JSON.stringify({ type: "input_audio_buffer.append", audio: chunk.toString("base64") }));

import json, base64, websocket

ws = websocket.create_connection(
    "wss://api.onysoft.com/v1/realtime?model=openai/gpt-realtime-1.5",
    header=["Authorization: Bearer sk-ony-..."],
)

while True:
    e = json.loads(ws.recv())

    if e["type"] == "session.created":
        ws.send(json.dumps(session_update))

    elif e["type"] == "response.output_audio.delta":
        hoparlore_yaz(base64.b64decode(e["delta"]))

    elif e["type"] == "response.done":
        print(e["response"]["usage"])   # token dokumu

    elif e["type"] == "error":
        print("HATA:", e["error"]["message"])

Interruptions (Barge-in)

In a real conversation the user cuts the model off. Handle it badly and two voices overlap. The correct behaviour: the moment you receive input_audio_buffer.speech_started, stop playback, flush your queue and send response.cancel. With server_vad on, the server starts the new turn by itself.

Function Calling

If the assistant needs to look up an order, book an appointment or write into a CRM, declare tools inside session.update. The model signals a call with response.function_call_arguments.done; you run it, return the result with conversation.item.create, then send response.create so the model continues.

Duration and Limits

Billing

You are charged on actual token usage. When a session closes, the input and output tokens it consumed are deducted from your balance; there is no separate charge for connection time. Audio tokens cost noticeably more than text tokens, because one second of audio carries far more tokens than a few words of text.

Error Codes

Common Mistakes

• Sending audio without base64 encoding — the server cannot decode it and the audio is effectively lost.
• Not matching input and output format to your line; pcm16 on a phone line produces crackle.
• Not handling interruptions, so two voices overlap when the user speaks over the model.
• Keeping a session open to the 30-minute limit without planning a reconnect.
• Embedding the key in a browser. Always open the connection from your own server.