NVIDIA Nemotron 3.5 Transcribe: 40 Languages, Speaker Labels, and Local Setup

What the 30B-A4B Model Covers: 40 Languages, Code-Switching, and Overlapping Speakers

For the last few years, getting a speaker-labeled, translated meeting transcript locally meant building a fragile pipeline. You ran an acoustic model like Whisper or Parakeet to get the raw words, ran a second model like pyannote to cluster voices by timestamp, wrote a script to align the two outputs, and then passed the text to a local or cloud LLM for translation. Whenever two people talked at the same time during a heated meeting, the timestamp alignment drifted and attributed half a sentence to the wrong person.

Nemotron 3.5 Transcribe folds all of those stages into one model by pairing a Conformers-based audio encoder with a 30B-A4B language decoder and a dedicated Connectionist Temporal Classification (CTC) timestamp head (ctc_timestamp.*, configured via word_timestamp_config in the checkpoint). Instead of treating speaker changes as an acoustic clustering problem bolted on after transcription, the model outputs speaker tags, punctuation, and word timing directly alongside the text across recordings up to 60 minutes long.

Language support covers 40 languages out of the box, covering major European, Asian, and Middle Eastern languages alongside built-in bidirectional translation. Where the 30B language backbone makes a noticeable difference over standard acoustic models is cross-lingual code-switching—recordings where bilingual speakers naturally blend two languages inside the same sentence, such as Hindi-English or Spanish-English business calls. Standard speech recognizers usually force you to lock in a single language tag per segment or constantly drop into gibberish when a speaker switches from Hindi grammar to English technical nouns. Because Nemotron 3.5 Transcribe routes audio tokens through a 30B multilingual language model, it keeps the word error rate low across mid-sentence switches and handles complex medical or engineering terminology without spelling words phonetically.

That large language backbone comes with trade-offs when you quantize the weights down to fit on consumer hardware. In my testing with 4-bit quantized weights (Q4 and IQ4_NL), the model handles multi-speaker conversation and technical vocabulary well, but its single-pass embedded translation at 4-bit is still stiffer and more prone to phrasing errors than taking a clean transcript and feeding it to a dedicated LLM. It can also trip over music mixed with speech. Running a full anime episode through a 4-bit checkpoint—both on the raw audio track and after isolating vocals—failed three times in a row because English lyrics in the opening theme song threw off the decoder's language state for the dialogue that followed.

Nemotron 3.5 Transcribe vs. Standard Local Models and Parakeet

If you already use local transcription tools—whether through a command-line script or an app like MacWhisper—your main question is whether a 30B model is worth the jump in memory over standard local models or NVIDIA's own Parakeet family.

For straightforward, single-speaker English or European-language audio, Parakeet TDT v3 is still faster, lighter, and often more accurate than a 4-bit quantized run of Nemotron 3.5 Transcribe. Parakeet takes roughly 1 GB of memory, runs fast even on a CPU, and never struggles to fit alongside your other apps. That memory gap is the exact reason lightweight desktop captioning and subtitling tools—such as Subtitle Edit—continue to recommend Parakeet rather than integrating a 30B model whose decoder alone takes 15 to 20 GB of memory.

Against a tuned Whisper large-v3 stack on a Mac, Nemotron 3.5 Transcribe is noticeably slower and heavier if all you are doing is transcribing a clean podcast or voice memo. You should reach for Nemotron 3.5 Transcribe specifically when Whisper and Parakeet break down: recordings where three people talk over one another in a meeting room, bilingual interviews with constant code-switching, or domain-heavy recordings (like medical dictation or architecture reviews) where a 1B acoustic model guesses the wrong homophone every thirty seconds.

Hardware and VRAM Requirements: The 30B MoE Memory Trap

When people read "30B parameters with 4B active," it is tempting to assume the model runs on an 8 GB or 12 GB graphics card like a standard 4B model. In an MoE architecture, the 4B active count dictates how much compute happens per token, which keeps inference speed respectable once loaded. However, the router can pick different experts for every single token, meaning all 30 billion parameters still have to sit in VRAM or unified system memory at the same time. If the weights spill over onto your SSD, token generation grinds to a crawl.

Here is how much disk and memory space each format actually requires:

First, the full-precision BF16 checkpoint (nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B) weighs roughly 67.5 to 68 GB. Unless you are fine-tuning the model on a multi-GPU server node, there is no reason to download the BF16 weights for daily transcription. Even on a 128 GB workstation like an NVIDIA DGX Spark (GB10), dedicating 68 GB to the transcription model alone leaves too little headroom if you also want to keep a local LLM or voice agent loaded beside it.

Second, the official quantized NVFP4 (NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-NVFP4) and FP8 checkpoints are the ones NVIDIA intends for local deployment on modern GPUs. If you checked Hugging Face right after the initial announcement and saw 404 errors on the NVFP4 and FP8 links while only BF16 and the DFlash2 speculative decoding drafter were live, those repositories have since been published. The NVFP4 checkpoint shrinks the model down to 21.6 GB, allowing it to fit on a single 24 GB GPU (such as an RTX 3090, RTX 4090, or RTX 5090) for shorter clips, though hour-long recordings with full context comfortably push total VRAM usage into the 24 GB to 32 GB bracket.

Third, for Apple Silicon Macs and local C++ setups using GGUF (IQ4_NL + F16), the model is split into two files: a 4-bit quantized language decoder (NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-AD-IQ4_NL.gguf) and a 16-bit audio encoder (Encoder-F16.gguf). Together, those two files take about 21.35 GB on disk. When loaded on an Apple Silicon Mac (such as an M-series Max with 36 GB or 64 GB of unified memory), the active footprint sits right around 20 GB of RAM.

If you have a 12 GB GPU like an RTX 3060 or RTX 4070 and hope to offload the inactive MoE experts to system RAM the way you would in llama.cpp, watch out for a current limitation in NeMo-Speech.cpp. Passing a flag like --asr.salm.n_cpu_moe 40 to nemo-speech serve immediately fails with nemo-speech serve: unknown option: --asr.salm.n_cpu_moe. Until upstream adds an MoE CPU-offload flag or splits the GGUF cleanly across two 16 GB cards, cards with 12 GB or 16 GB of VRAM cannot hold both the decoder and the F16 encoder in GPU memory without custom patches.

How to Run Nemotron 3.5 Transcribe Locally via /v1/audio/transcriptions

Once you have a machine with at least 24 GB to 32 GB of VRAM (for Linux + NVIDIA GPUs) or 32 GB+ of unified memory

For NVIDIA GPUs, SGLang-Omni provides day-one support for serving the model with high throughput. For quantized GGUF weights on Mac or desktop workstations, NeMo-Speech.cpp exposes a local server through nemo-speech serve. Here is the step-by-step workflow to get your local endpoint running and transcribe a multi-speaker recording:

1. Download the quantized weights instead of BF16. If you are running SGLang-Omni on a 24 GB+ NVIDIA GPU, pull nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-NVFP4 (or the -FP8 variant) from Hugging Face. If you are using NeMo-Speech.cpp on a Mac with 32 GB+ of unified memory, download both the NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-AD-IQ4_NL.gguf decoder file and the Encoder-F16.gguf audio encoder file.
2. Prepare your meeting audio file. Convert your saved Zoom, Teams, or recorder file into a standard 16 kHz mono .wav or .mp3 file (ffmpeg -i meeting.m4a -ar 16000 -ac 1 meeting.wav). Keeping individual files under 60 minutes ensures you stay within the model's trained context window and prevents KV-cache memory spikes.
3. Start the local transcription server. With SGLang-Omni installed in your Python environment, launch the server pointing at the checkpoint on port 8000 (following the launch arguments in the SGLang-Omni Nemotron Transcribe cookbook). If you are using NeMo-Speech.cpp with GGUF files instead, run nemo-speech serve --asr-model NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-AD-IQ4_NL.gguf along with the F16 encoder path and wait until the ~20 GB model finishes mapping into memory.
4. Send the audio file to /v1/audio/transcriptions. Once the server is listening locally, send a POST request using curl or a standard Python HTTP client to generate a speaker-labeled transcript:

curl http://localhost:8000/v1/audio/transcriptions \
  -H "Content-Type: multipart/form-data" \
  -F file="@meeting.wav" \
  -F model="nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-NVFP4" \
  -F response_format="verbose_json" \
  -F timestamp_granularities[]="word" \
  -F timestamp_granularities[]="segment"

If you want the model to translate a multilingual or code-switched recording into another target language during the same pass, pass the target language parameter in the request body or prompt configuration supported by your serving backend, and save the returned JSON output to disk.

Two Local Bugs to Watch Out For on AMD ROCm and Apple Silicon

If you deviate from a standard CUDA or NeMo-Speech.cpp Mac setup, there are two platform-specific issues you can hit right now that look like hardware failures if you do not know what is causing them.

First, if you compile NeMo-Speech.cpp with HIP/ROCm for an AMD GPU (such as a Radeon AI PRO R9700 or RX 7900 XTX on ROCm 7.x), short audio clips of 30 or 60 seconds will transcribe fine, but any audio file longer than roughly 120 to 160 seconds will hang indefinitely. When this happens, your GPU utilization drops to 0% while a single CPU thread spins at 100% inside hipStreamSynchronize, and nemo-speech serve locks up on subsequent requests. This happens because NeMo-Speech.cpp pins an older llama.cpp submodule commit from before the HIP ARGSORT fix was merged upstream; when audio length pushes the sorting matrix past 1,024 columns on a ROCm build without CUB, execution stalls. Updating or patching the underlying ggml/llama.cpp submodule to include the upstream HIP ARGSORT fix resolves the hang on files over two minutes.

Second, if you are on a Mac and try to run the model through mlx-audio instead of NeMo-Speech.cpp, do not expect .srt or .vtt subtitle exports to work yet. Even though the public Hugging Face weights include the ctc_timestamp.* head for word-level timing, mlx-audio currently drops the ctc_timestamp.* weights when loading the model. Because those timestamp weights are stripped at load time, mlx-audio can only output plain .txt or basic .json transcripts. If your end goal is generating timed SRT or VTT caption files with speaker labels, stick to SGLang-Omni or NeMo-Speech.cpp so the CTC timestamp head stays intact.