Soniox STT v6: 60+ Language Transcription, Translation & Echo Cancellation

What Changed in stt-rt-v6 for 60+ Language Code-Switching and Formatting

The strongest reason I reach for Soniox over single-language streaming engines is how it handles multilingual code-switching inside a single sentence. When I dictate technical notes or talk in meetings where sentences mix Mandarin, English, and French—or jump between Spanish and English without warning—stt-rt-v6 switches scripts on the fly without requiring a language toggle or separate audio routing. Over a stable connection, end-to-end streaming latency sits between 500 milliseconds and 1 second for the first token to appear.

In v6, spoken numbers, currency amounts, and alphanumeric product codes come back formatted much closer to how you would naturally type them, rather than spelled-out words. However, v6 changes how you handle voice dictation commands. If you say formatting instructions out loud like period, new line, or scratch that, the model transcribes those phrases literally as words instead of converting them into punctuation marks or deleting the previous clause. If you use stt-rt-v6 inside a custom voice dictation tool, your client application now has to parse those literal command phrases from the token stream and execute the text edits itself.

It also helps to know where the 60-language ceiling stops. While Soniox covers major European, Asian, and Middle Eastern languages in one model, it does not cover every regional spoken language. Languages including Nepali, Sinhala, Khmer, Burmese, Lao, Amharic, Armenian, Georgian, Mongolian, Icelandic, Irish, Javanese, Yoruba, Zulu, Hausa, and Somali are missing from the supported language list. If a speaker uses one of those languages, you cannot select it in the language configuration or get native script transcription from Soniox, so you will still need a fallback provider for those specific locales.

How Agent Text Input and Voice Echo Cancellation Work Without Headphones

The biggest operational headache with live voice agents and desktop captioning is acoustic echo. If you play an AI assistant's voice or a remote meeting participant through your laptop speakers without headphones, your microphone picks up that speaker output and feeds it right back into the speech-to-text stream. The transcription engine ends up transcribing the computer's own voice as if you said it, which triggers false interruptions and ruins turn-taking.

Soniox STT v6 addresses this inside the real-time model through Agent Text Input and Agent Voice Echo Cancellation. Instead of relying purely on hardware acoustic echo cancellation at the OS level, you feed the exact text that your voice agent is speaking into the active stt-rt-v6 session via Agent Text Input. Because the transcription engine now knows the text content of the audio coming out of the speakers, Agent Voice Echo Cancellation strips that matching speech out of the incoming microphone stream while still listening for your voice. You can leave your speakers unmuted and interrupt the agent mid-sentence, and the stream only transcribes what you actually say.

There is an important architectural boundary here that determines whether you can use this feature right away:

• Voice agents with known reply scripts: If you are building an interactive AI voice agent, your backend generates the LLM text reply before or while the text-to-speech engine plays it aloud. You already have the exact reply text in memory, so passing it into Agent Text Input works out of the box.
• Voice dictation and two-way human meetings: If you are building a standalone voice dictation app or recording a two-way Zoom or Teams call on speakerphone, your app does not have a pre-written script of what the other human on the call is about to say. For standard dictation, Agent Text Input does nothing and should be left off. For speakerphone meetings without headphones, the only way to use Soniox's echo cancellation is a two-channel workaround: you run one real-time stream to transcribe the system audio coming from the remote speaker, and simultaneously feed that live transcript as Agent Text Input into the microphone stream to suppress the room bleed.

Live Translation Limits and the Speaker Diarization Context Trap

Soniox STT v6 lets you run real-time translation inside the same streaming WebSocket connection as your transcription, which is convenient when you want low-latency subtitles without wiring up a second API. When I test and deploy this in real workflows, two behavior limits stand out across translation and speaker diarization.

First, if you configure one_way translation in the Soniox client, the target_language field only accepts a single language string. If you pass an array or multiple target languages hoping to broadcast live Spanish, French, and Japanese captions from one English stream, the API silently drops the extra languages and only outputs the first one. To output multiple translated languages at the same time for a live webinar or event, you have to open a separate WebSocket session for each target language. Furthermore, while stt-rt-v6 is accurate at transcribing source audio—especially when you feed domain terminology and hotwords into the context parameter—its built-in translation reads noticeably stiffer than a dedicated large language model when handling slang or fast live-stream banter. When translation fluency matters more than saving 300 milliseconds, I use stt-rt-v6 strictly for transcription and pipe the finalized sentences into an external LLM with rolling conversation context.

Second, if you turn on speaker diarization (enable_speaker_diarization: true) to label who is speaking, you have to watch the character length of your context.text payload. Even though the Soniox documentation permits up to 8,000 tokens of context, passing a context.text string around 3,000 characters long causes speaker diarization to silently stop working: every returned token comes back with speaker: null instead of "1" or "2", with no error code thrown. Keeping each context block trimmed to around 400 characters—roughly 800 characters total for domain terms and participant names—preserves the accuracy boost from hotwords while keeping speaker labels intact. Also keep in mind that while Soniox aligns word timestamps accurately when two people talk, rapid back-and-forth interruptions with zero pause between speakers can still merge both voices into a single unbroken sentence block.

Cutting 250ms of Latency and Fixing WebSocket 401 Errors

When you wire stt-rt-v6 into a real-time application, most of the friction comes from WebSocket session management rather than model accuracy. Adjusting three connection settings prevents dropped connections and shaves noticeable lag off every turn.

If you run a local Voice Activity Detector (VAD) in a framework like LiveKit to detect when the speaker stops talking, simply flushing the audio stream leaves you waiting for Soniox's server-side endpointing to decide the sentence is over. In practice, waiting for that automatic endpoint takes between 664 and 1,058 milliseconds after the last spoken word. If you explicitly send the JSON control message {"type": "finalize"} over the WebSocket the moment your VAD detects silence, the final transcript arrives in 440 to 475 milliseconds—cutting roughly 250 milliseconds off every single response.

You also need to check how your client authenticates and which regional hostname it hits. Historically, Soniox WebSocket connections (wss://stt-rt.soniox.com/transcribe-websocket) accepted the api_key inside the first JSON configuration frame after opening the socket. Starting January 15, 2027, connections that only pass the API key inside that initial JSON message will be rejected with 401 unauthenticated. You must pass Authorization: Bearer <YOUR_API_KEY> in the HTTP handshake headers when opening the WebSocket. Moving authentication to the handshake header also makes it easier to route traffic through unified API gateways like LiteLLM, which previously could not proxy raw Soniox WebSocket streams because the key was buried inside the first payload frame unless you mapped through a GPT realtime wrapper.

Finally, if your valid API key suddenly returns a 401 error, check your project region. Soniox scopes keys to regional data centers, and an EU project key will always return 401 if your app hard-codes the default US host (stt-rt.soniox.com or api.soniox.com). You have to match the endpoint to your project's region: stt-rt.eu.soniox.com and api.eu.soniox.com for the European Union, stt-rt.jp.soniox.com for Japan, and stt-rt.in.soniox.com for India. Routing an European client directly to the .eu host also cuts WebSocket connection setup time from around 0.55 seconds down to 0.30 seconds.

Soniox STT v6 vs. Whisper, Parakeet, and Nemotron 3.5 Transcribe

Choosing between Soniox STT v6 and other current models comes down to whether you need cloud-based multilingual code-switching, local offline privacy, or heavy multi-speaker overlap handling.

When I test voice dictation across languages, Whisper (whether running on Groq's fast HTTP API or locally inside MacWhisper) still sets a high bar for pure English batch files, and Apple Speech is convenient for offline macOS use. However, both stumble when I mix Mandarin, English, and French in the same breath—either transliterating English code terms into phonetic characters or locking onto the wrong language for the rest of the sentence. If you only dictate in English and want instant local response without paying per-minute API fees, running the 160 MB Parakeet streaming model inside Hex 3.0 local voice dictation gives you 100 ms GPU streaming right on your machine. And if your priority is transcribing heavy multi-speaker medical or legal recordings on your own hardware with speaker labels, NVIDIA Nemotron 3.5 Transcribe handles 40 languages and overlapping voices locally if you have the VRAM to host a 30B-A4B MoE model.

Soniox STT v6 wins when you cannot predict which of the 60+ languages a speaker will use next, or when you are building a voice agent that needs to cancel out its own speakerphone output without forcing the user to put on AirPods.

Of course, Soniox STT v6 is a developer API, not a desktop application with a caption window. If you just want to read live captions and bilingual translations during a multilingual Zoom call, university lecture, or livestream without writing WebSocket code or managing regional API keys, I use Vocena. Unlike cloud meeting bots that join your call as an extra participant, or single-language dictation utilities, Vocena runs directly on your computer, captures both system audio and your microphone simultaneously, and displays real-time captions and translations on screen—sitting in a similar bot-free desktop category as the Otter Mac desktop app, but built around multilingual live translation and dual-language reading. Vocena is best for remote workers, students, and international teams who deal with mixed-language audio every day; it offers a free plan that covers basic live captioning and short daily sessions, while unlimited continuous real-time translation, long meeting recording, and full transcript exports require a paid subscription.