# NVIDIA Nemotron 3.5 Transcribe: 40 Languages, Speaker Labels, and Local Setup

> See what NVIDIA Nemotron 3.5 Transcribe (30B-A4B) actually does across 40 languages and overlapping speakers, how it compares to Whisper and Parakeet, how much VRAM you need.

- URL: https://freelivecaption.com/en/posts/nemotron-3-5-transcribe
- Author: Emily Carter (https://freelivecaption.com/en/about)
- Topic: Dictation (https://freelivecaption.com/en/topics/dictation)

## Summary

- NVIDIA Nemotron 3.5 Transcribe (30B-A4B) combines 40-language transcription, overlapping speaker diarization, word-level timestamps.
- Although only 4B parameters activate per token, all 30B weights must stay in memory: 67.5 GB in BF16, 21.6 GB in NVFP4.
- Compared to Parakeet TDT v3 (~1 GB VRAM) and Whisper large-v3, Nemotron 3.5 Transcribe is heavier and slower on simple single-speaker clips.
- You can serve the model locally via SGLang-Omni or NeMo-Speech.cpp and process meeting recordings through a standard /v1/audio/transcriptions endpoint.

---

When NVIDIA released the open weights for [NVIDIA-Nemotron-3.5-Transcribe-30B-A4B](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B), the pitch immediately caught my attention: a single model that handles speech transcription across 40 languages, labels multiple speakers even when they talk over each other, generates word-level timestamps, and translates audio without chaining three separate tools together. Under the hood, it is a 30-billion-parameter Speech-Audio Language Model (SALM) built on a hybrid Mixture-of-Experts (MoE) architecture that activates roughly 4 billion parameters per token.

Before you start downloading 20 to 68 gigabytes of weights, there is one major boundary you need to know upfront. Nemotron 3.5 Transcribe only processes recorded audio files—such as saved meeting recordings, interviews, or call exports up to an hour long—and does not support real-time microphone streaming.

It is easy to mix this up because NVIDIA also maintains a completely separate, lightweight model named `nemotron-3.5-asr-streaming-0.6b` that powers low-latency dictation tools on consumer laptops. If you want live captions appearing on screen while someone is speaking in a room or on a call, this 30B model is not built for that workflow, and you are better off using one of the dedicated [live caption apps for Mac](/en/posts/live-caption-apps-mac) or a streaming 0.6B model. If what you have is a finished recording where people interrupt each other, switch languages mid-sentence, or use specialized terminology, Nemotron 3.5 Transcribe is designed to process that file in one pass.

## What the 30B-A4B Model Covers: 40 Languages, Code-Switching, and Overlapping Speakers

For the last few years, getting a speaker-labeled, translated meeting transcript locally meant building a fragile pipeline. You ran an acoustic model like Whisper or Parakeet to get the raw words, ran a second model like `pyannote` to cluster voices by timestamp, wrote a script to align the two outputs, and then passed the text to a local or cloud LLM for translation. Whenever two people talked at the same time during a heated meeting, the timestamp alignment drifted and attributed half a sentence to the wrong person.

Nemotron 3.5 Transcribe folds all of those stages into one model by pairing a Conformers-based audio encoder with a 30B-A4B language decoder and a dedicated Connectionist Temporal Classification (CTC) timestamp head (`ctc_timestamp.*`, configured via `word_timestamp_config` in the checkpoint). Instead of treating speaker changes as an acoustic clustering problem bolted on after transcription, the model outputs speaker tags, punctuation, and word timing directly alongside the text across recordings up to 60 minutes long.

Language support covers 40 languages out of the box, covering major European, Asian, and Middle Eastern languages alongside built-in bidirectional translation. Where the 30B language backbone makes a noticeable difference over standard acoustic models is cross-lingual code-switching—recordings where bilingual speakers naturally blend two languages inside the same sentence, such as Hindi-English or Spanish-English business calls. Standard speech recognizers usually force you to lock in a single language tag per segment or constantly drop into gibberish when a speaker switches from Hindi grammar to English technical nouns. Because Nemotron 3.5 Transcribe routes audio tokens through a 30B multilingual language model, it keeps the word error rate low across mid-sentence switches and handles complex medical or engineering terminology without spelling words phonetically.

That large language backbone comes with trade-offs when you quantize the weights down to fit on consumer hardware. In my testing with 4-bit quantized weights (`Q4` and `IQ4_NL`), the model handles multi-speaker conversation and technical vocabulary well, but its single-pass embedded translation at 4-bit is still stiffer and more prone to phrasing errors than taking a clean transcript and feeding it to a dedicated LLM. It can also trip over music mixed with speech. Running a full anime episode through a 4-bit checkpoint—both on the raw audio track and after isolating vocals—failed three times in a row because English lyrics in the opening theme song threw off the decoder's language state for the dialogue that followed.

## Nemotron 3.5 Transcribe vs. Standard Local Models and Parakeet

If you already use local transcription tools—whether through a command-line script or an app like [MacWhisper](/en/posts/macwhisper-guide)—your main question is whether a 30B model is worth the jump in memory over standard local models or NVIDIA's own Parakeet family.

| Feature / Spec | Nemotron 3.5 Transcribe (30B-A4B) | NVIDIA Parakeet TDT v3 | Whisper large-v3 / large-v3-turbo |
| :--- | :--- | :--- | :--- |
| **Total Parameters** | 30B (approx. 4B active per token) | 0.6B to 1.1B | 1.55B (large-v3) / 0.8B (turbo) |
| **Typical Memory (VRAM / Unified RAM)** | 20 GB – 32 GB (quantized) / 68 GB (BF16) | ~1 GB – 2 GB | ~1.5 GB – 4 GB |
| **Supported Languages** | 40 languages + code-switching | 25 European languages (v3) / English | 99 languages |
| **Native Speaker Diarization** | Yes (including overlapping cross-talk) | No (requires external diarization) | No (requires pyannote / WhisperX) |
| **Word-Level Timestamps** | Yes (via built-in `ctc_timestamp` head) | Yes (Token-and-Duration Transducer) | Approximation via cross-attention |
| **Built-In Translation** | Multi-language translation in one pass | No | Translate-to-English only |
| **Best Use Case** | Multi-speaker meetings, cross-talk, medical/technical terms, code-switching | Fast everyday transcription, CPU/low-VRAM machines, subtitle editors | Broad 99-language coverage on laptops with 8 GB – 16 GB RAM |

For straightforward, single-speaker English or European-language audio, Parakeet TDT v3 is still faster, lighter, and often more accurate than a 4-bit quantized run of Nemotron 3.5 Transcribe. Parakeet takes roughly 1 GB of memory, runs fast even on a CPU, and never struggles to fit alongside your other apps. That memory gap is the exact reason lightweight desktop captioning and subtitling tools—such as Subtitle Edit—continue to recommend Parakeet rather than integrating a 30B model whose decoder alone takes 15 to 20 GB of memory.

Against a tuned Whisper large-v3 stack on a Mac, Nemotron 3.5 Transcribe is noticeably slower and heavier if all you are doing is transcribing a clean podcast or voice memo. You should reach for Nemotron 3.5 Transcribe specifically when Whisper and Parakeet break down: recordings where three people talk over one another in a meeting room, bilingual interviews with constant code-switching, or domain-heavy recordings (like medical dictation or architecture reviews) where a 1B acoustic model guesses the wrong homophone every thirty seconds.

## Hardware and VRAM Requirements: The 30B MoE Memory Trap

When people read "30B parameters with 4B active," it is tempting to assume the model runs on an 8 GB or 12 GB graphics card like a standard 4B model. In an MoE architecture, the 4B active count dictates how much compute happens per token, which keeps inference speed respectable once loaded. However, the router can pick different experts for every single token, meaning all 30 billion parameters still have to sit in VRAM or unified system memory at the same time. If the weights spill over onto your SSD, token generation grinds to a crawl.

Here is how much disk and memory space each format actually requires:

First, the full-precision **BF16 checkpoint (`nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B`)** weighs roughly 67.5 to 68 GB. Unless you are fine-tuning the model on a multi-GPU server node, there is no reason to download the BF16 weights for daily transcription. Even on a 128 GB workstation like an NVIDIA DGX Spark (GB10), dedicating 68 GB to the transcription model alone leaves too little headroom if you also want to keep a local LLM or voice agent loaded beside it.

Second, the official quantized **NVFP4 (`NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-NVFP4`) and FP8 checkpoints** are the ones NVIDIA intends for local deployment on modern GPUs. If you checked Hugging Face right after the initial announcement and saw 404 errors on the NVFP4 and FP8 links while only BF16 and the `DFlash2` speculative decoding drafter were live, those repositories have since been published. The NVFP4 checkpoint shrinks the model down to 21.6 GB, allowing it to fit on a single 24 GB GPU (such as an RTX 3090, RTX 4090, or RTX 5090) for shorter clips, though hour-long recordings with full context comfortably push total VRAM usage into the 24 GB to 32 GB bracket.

Third, for Apple Silicon Macs and local C++ setups using **GGUF (`IQ4_NL` + `F16`)**, the model is split into two files: a 4-bit quantized language decoder (`NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-AD-IQ4_NL.gguf`) and a 16-bit audio encoder (`Encoder-F16.gguf`). Together, those two files take about 21.35 GB on disk. When loaded on an Apple Silicon Mac (such as an M-series Max with 36 GB or 64 GB of unified memory), the active footprint sits right around 20 GB of RAM.

If you have a 12 GB GPU like an RTX 3060 or RTX 4070 and hope to offload the inactive MoE experts to system RAM the way you would in `llama.cpp`, watch out for a current limitation in `NeMo-Speech.cpp`. Passing a flag like `--asr.salm.n_cpu_moe 40` to `nemo-speech serve` immediately fails with `nemo-speech serve: unknown option: --asr.salm.n_cpu_moe`. Until upstream adds an MoE CPU-offload flag or splits the GGUF cleanly across two 16 GB cards, cards with 12 GB or 16 GB of VRAM cannot hold both the decoder and the F16 encoder in GPU memory without custom patches.

## How to Run Nemotron 3.5 Transcribe Locally via `/v1/audio/transcriptions`

Once you have a machine with at least 24 GB to 32 GB of VRAM (for Linux + NVIDIA GPUs) or 32 GB+ of unified memory

For NVIDIA GPUs, [SGLang-Omni](https://sgl-project.github.io/sglang-omni/cookbook/nemotron_transcribe.html) provides day-one support for serving the model with high throughput. For quantized GGUF weights on Mac or desktop workstations, `NeMo-Speech.cpp` exposes a local server through `nemo-speech serve`. Here is the step-by-step workflow to get your local endpoint running and transcribe a multi-speaker recording:

1. **Download the quantized weights instead of BF16.** If you are running SGLang-Omni on a 24 GB+ NVIDIA GPU, pull `nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-NVFP4` (or the `-FP8` variant) from Hugging Face. If you are using `NeMo-Speech.cpp` on a Mac with 32 GB+ of unified memory, download both the `NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-AD-IQ4_NL.gguf` decoder file and the `Encoder-F16.gguf` audio encoder file.
2. **Prepare your meeting audio file.** Convert your saved Zoom, Teams, or recorder file into a standard 16 kHz mono `.wav` or `.mp3` file (`ffmpeg -i meeting.m4a -ar 16000 -ac 1 meeting.wav`). Keeping individual files under 60 minutes ensures you stay within the model's trained context window and prevents KV-cache memory spikes.
3. **Start the local transcription server.** With SGLang-Omni installed in your Python environment, launch the server pointing at the checkpoint on port `8000` (following the launch arguments in the [SGLang-Omni Nemotron Transcribe cookbook](https://sgl-project.github.io/sglang-omni/cookbook/nemotron_transcribe.html)). If you are using `NeMo-Speech.cpp` with GGUF files instead, run `nemo-speech serve --asr-model NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-AD-IQ4_NL.gguf` along with the F16 encoder path and wait until the ~20 GB model finishes mapping into memory.
4. **Send the audio file to `/v1/audio/transcriptions`.** Once the server is listening locally, send a `POST` request using `curl` or a standard Python HTTP client to generate a speaker-labeled transcript:

```bash
curl http://localhost:8000/v1/audio/transcriptions \
  -H "Content-Type: multipart/form-data" \
  -F file="@meeting.wav" \
  -F model="nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-NVFP4" \
  -F response_format="verbose_json" \
  -F timestamp_granularities[]="word" \
  -F timestamp_granularities[]="segment"
```

If you want the model to translate a multilingual or code-switched recording into another target language during the same pass, pass the target language parameter in the request body or prompt configuration supported by your serving backend, and save the returned JSON output to disk.

## Two Local Bugs to Watch Out For on AMD ROCm and Apple Silicon

If you deviate from a standard CUDA or `NeMo-Speech.cpp` Mac setup, there are two platform-specific issues you can hit right now that look like hardware failures if you do not know what is causing them.

First, if you compile `NeMo-Speech.cpp` with HIP/ROCm for an AMD GPU (such as a Radeon AI PRO R9700 or RX 7900 XTX on ROCm 7.x), short audio clips of 30 or 60 seconds will transcribe fine, but any audio file longer than roughly 120 to 160 seconds will hang indefinitely. When this happens, your GPU utilization drops to 0% while a single CPU thread spins at 100% inside `hipStreamSynchronize`, and `nemo-speech serve` locks up on subsequent requests. This happens because `NeMo-Speech.cpp` pins an older `llama.cpp` submodule commit from before the HIP `ARGSORT` fix was merged upstream; when audio length pushes the sorting matrix past 1,024 columns on a ROCm build without CUB, execution stalls. Updating or patching the underlying `ggml`/`llama.cpp` submodule to include the upstream HIP `ARGSORT` fix resolves the hang on files over two minutes.

Second, if you are on a Mac and try to run the model through `mlx-audio` instead of `NeMo-Speech.cpp`, do not expect `.srt` or `.vtt` subtitle exports to work yet. Even though the public Hugging Face weights include the `ctc_timestamp.*` head for word-level timing, `mlx-audio` currently drops the `ctc_timestamp.*` weights when loading the model. Because those timestamp weights are stripped at load time, `mlx-audio` can only output plain `.txt` or basic `.json` transcripts. If your end goal is generating timed [SRT or VTT](/en/posts/srt-vs-vtt) caption files with speaker labels, stick to SGLang-Omni or `NeMo-Speech.cpp` so the CTC timestamp head stays intact.

## Sources and checks

<a id="source-huggingface-co-1"></a>

- [NVIDIA-Nemotron-3.5-Transcribe-30B-A4B](<https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B>) — Checked 2026-11-01

<a id="source-sgl-project-github-io-2"></a>

- [SGLang-Omni](<https://sgl-project.github.io/sglang-omni/cookbook/nemotron_transcribe.html>) — Checked 2026-11-01