Twilio Conversation Relay

Executive Summary

Twilio Conversation Relay is a managed speech layer for live calls. It transcribes the caller, sends text to an application's WebSocket server and speaks the text that server returns. The application supplies its language model and conversational logic.^[1] It fits the agentic voice APIs comparison as phone infrastructure for a custom agent, rather than as a standalone conversational model.

This division is useful when a team already owns its reasoning, retrieval or business workflows and wants a managed path to telephone speech. It also makes the engineering boundary explicit: connecting a number does not supply authorization, a reliable tool workflow or the state needed after an interrupted answer.

Connection and Application Architecture

The onboarding flow requires a Twilio account, a reachable secure WebSocket server and TwiML that places ConversationRelay inside Connect. The application also accepts the product's AI features terms. Twilio includes a signature in the initial WebSocket handshake; the receiving server validates it using the account credentials and request URL before accepting the connection.^[2]

A typical integration has four responsibilities:

The product overview's speech-to-text → application → text-to-speech flow establishes this separation.^[1] A custom server can use an LLM, deterministic application logic or a combination. The application remains the place to validate an account change before a speech response tells the caller it succeeded.

Speech and Language Choices

The current TwiML reference lists Google and Deepgram for recognition, with Google, Amazon and ElevenLabs for synthesis. Language-specific configurations can select a voice and speech model. Automatic multilingual mode has provider constraints: Deepgram for recognition and ElevenLabs for synthesis.^[3] Provider choice and the selected language should be part of a reproducible test configuration.

Twilio recommends streaming model text as it arrives and marking the final chunk, while retaining meaningful spacing. Its guide also recommends normalizing dates, currency, abbreviations and similar strings for speech. Per-language voice configurations should be set before the session; switching the active language can otherwise select an unintended default voice.^[4]

For an order-status assistant, the important test is whether the caller hears the correct date, amount and tracking identifier. A natural voice delivering an ambiguous date is still a failed interaction. Test the actual vocabulary and phone audio of the application, including unusual names and noisy calls.

Interruptions, Handoff and Recovery

Two settings govern different behavior. interruptible controls whether caller input stops playback; reportInputDuringAgentSpeech controls whether the application receives input while the agent speaks. The documented defaults are any for the former and none for the latter.^[3] Configure them deliberately instead of treating “barge-in enabled” as a complete conversation policy.

The WebSocket interruption event includes the utterance spoken before interruption and elapsed playback time. The protocol also provides setup, prompt, DTMF and error messages. An application can end the Relay session with handoff data that returns through the Connect action callback.^[5] This gives the application an opportunity to decide what happens next; an end message alone is not proof that a human answered.

The same guide explicitly says an unexpected WebSocket loss is not automatically reconnected. Its recovery recipe uses the Connect action callback to return a new Relay connection and checks that the call SID remains consistent.^[5] Preserve the business operation's state outside the socket so a reconnect cannot repeat a completed action or forget an unresolved request.

A worked interruption test

Consider a delivery-change assistant. It begins reading available dates when the caller says, “Stop—use my office address instead.” A useful pilot would:

1. Confirm playback stops and the corrected request reaches the application under the chosen input settings.
2. Cancel or supersede any obsolete model response and retain only the relevant spoken context.
3. Authenticate the address change and check the backend result before reporting success.
4. Break the socket after the backend commits. Recover the conversation and report the existing result without submitting the change again.
5. Ask for a person and test both a successful handoff and an unavailable destination.

These are proposed tests, not results from calling the service. They measure application behavior that a speech-synthesis quality score cannot establish.

Billing and Deployment Costs

Twilio's pricing page, marked current as of August 2026 and checked September 16, lists $0.07 per Conversation Relay minute and explicitly separates Voice costs.^[6] The integration guide defines Relay usage as active AI-agent session time.^[7]

For example, 1,000 metered Relay minutes at the published rate are $70 for Relay alone. That is arithmetic, not an all-inclusive cost for 1,000 customer calls. Model response length, carrier destination, call duration and recovery behavior can change the full bill.

The product is hosted usage access, not a downloadable speech-model license. A code sample can help explain the integration without changing the service's commercial terms. For capacity planning, test the account's allowed concurrent sessions and the application's ability to keep each connection healthy; the message reference includes a dedicated concurrency-limit error.^[5]

Data, Recording and Analysis

The observability guide says Twilio does not store Relay transcripts by default. An optional Conversation Intelligence classic integration persists transcripts and can apply language operators; it receives transcript text, not the call audio. That guide warns that the classic integration is not PCI compliant and rejects Relay transcripts when account PCI mode is enabled.^[7] Recording a call is a separate Voice capability described in the best-practices guide.^[4]

For Relay itself, Twilio qualifies PCI workflow support by the selected speech providers and excludes sensitive payment data from ordinary TwiML parameters and handoff fields.^[3] The application's model provider and its own logs are additional data-processing paths. No transcript persistence by default should not be paraphrased as a blanket guarantee that every related product stores nothing.

Developer Evidence and Fit

The deshartman/simple-conversation-relay project, linked by Twilio's overview, provides a useful reference implementation. Its September 3, 2026 v4.13 release documents fixes for overlapping speech, outbound callback handling, handoff and authentication of application endpoints.^[8] These are issues in that sample's integration and release history, not evidence of universal defects in the Relay service.

The practical lesson is that even a small application needs explicit ownership of streaming cancellation, call status and the endpoints that can spend account funds. Copying a demo without reviewing those controls is not the same as deploying the managed speech component correctly. This report reviewed documentation and the reference project's release notes; it did not execute the sample, place calls or measure latency.

Conversation Relay is a strong candidate for a Twilio-based team that wants to preserve its own model/tool layer. A team wanting a complete agent authoring and operations product should compare the surrounding work with Vapi, Retell AI and Bland. The decision turns on which responsibilities the team wants to retain, and whether the finished workflow survives interruptions, outages and failed handoffs.

---

Research by Ry Walker Research • methodology