How to Get a Speaker-Labelled Transcript From Two Models That Never Talk to Each Other
NVIDIA's new speaker model is 107 megabytes and the licence says your output is your own. The benchmark headline is the least interesting part.
NVIDIA published a model this week that answers one question and refuses to answer any others. Feed it a recording and it tells you that somebody talked from 1.8 seconds to 3.25, that somebody else talked from 1.8 to 4.1, and that those two people are different. It will not tell you what they said. It will not tell you who they are.
That sounds unhelpfully narrow until you notice what it costs. The whole thing is 99 million parameters, and there is a compressed copy sitting in the repository at 107 megabytes. The licence attached to it contains a sentence most model licences do not: it imposes no restrictions at all on how you use, change or share anything you produce with it.
A model that small, that permissive, doing one job well, is a building block. This article is about the thing you build with it: a transcript where every line knows who said it, assembled from two models that were never designed to work together and have no idea the other one exists.
The job this replaces
If you cut interviews, panels, podcasts or documentary sync, you already do this work. The transcript comes back as a wall of text. Every word is right and none of it is attributed. So you scrub. Voice change, type a name, scroll, again. On a two-hour panel with four people that is most of an afternoon, before the actual editing starts.
The technical word for splitting a recording by speaker is diarization, worth knowing only because it is what you type into a search box. What it means is: who spoke when.
NVIDIA Nemotron 3 Diarization does that and nothing else, for up to eight speakers. It handles people talking over each other, which is the case that breaks most approaches, because it can mark two people as active in the same instant instead of being forced to pick one.
The model files went up on September 1. The explanation, the demo and the ecosystem support all arrived on September 23, which is why it is worth your attention this week rather than three weeks ago.
Two models, one clock
Here is the part that makes the whole thing approachable, and it took me a while to see it.
The speaker model produces time ranges with labels: speaker_0 from 0.4 to 2.1, speaker_1 from 1.8 to 3.25, and so on. Separately, a transcription model produces words with times: "report" from 2.30 to 2.61.
Neither model knows the other exists. No integration, no shared format, no special version of either one. They are joined by the fact that both are measuring the same recording with the same stopwatch. For each word, take its midpoint, look at which speaker was active at that instant, write the label next to the word.
That is the entire trick, maybe fifteen lines of code, and NVIDIA publishes them. The practical consequence is that either half is replaceable. If your editing suite already exports a transcript with timecode, you do not need the transcription model at all. You need the speaker half and a join.
Put this into practice
Ten minutes, in a browser, no install.
- Open the live demo. It has four tabs: Conversation with prepared audio, Live Mic, Multilingual Live Mic, and Audio File.
- Skip the prepared conversations. They are synthetic and they flatter the model.
- Go to Audio File and upload fifteen minutes of your worst real recording. A panel with cross-talk, a room with air handling noise, a lav that was clipped badly. You are not testing whether it works. You are testing whether it works on your audio.
- Watch what it does at the moments where two people start at once. That is where you will spend your cleanup time, and knowing how much of it there is decides whether this is worth building on.
On a Mac or an iPhone, with no Python at all.
This is the route most people reading this should take, and it exists as of two days ago. Argmax shipped Pro SDK 3 on September 23 with day-zero support for this model, running on the device rather than in a data centre. They publish a free TestFlight build of a Playground app for Mac and iPhone, so you can test speaker separation on your own recordings without writing anything.
Argmax also made a claim in that post that is worth repeating because it inverts the usual order of operations. Until now their SDK assigned speakers to words after transcribing, to avoid one model's mistakes compounding the other's. With this model they now separate the speakers first and transcribe afterwards, and say it produces better transcripts in conversations with overlap. If you are building anything here, that ordering question is the one to test.
The full pipeline, if you have a Linux machine with an NVIDIA card.
Install the toolkit, then run the speaker model over a 16 kHz single-channel file:
from nemo.collections.asr.models import SortformerEncLabelModel
diar_model = SortformerEncLabelModel.from_pretrained("nvidia/Nemotron-3-Diarization")
diar_model.eval()
# The five numbers below are the offline preset. Set all five, never a mix.
diar_model.sortformer_modules.spkcache_len = 264
diar_model.sortformer_modules.fifo_len = 40
diar_model.sortformer_modules.chunk_len = 340
diar_model.sortformer_modules.chunk_right_context = 40
diar_model.sortformer_modules.spkcache_update_period = 300
diar_model._check_streaming_parameters()
segments = diar_model.diarize(audio=["/path/to/conversation.wav"], batch_size=1)
Those five numbers are not decoration and you cannot skip them. NVIDIA's instruction is to take all five from one row of their settings table and then call the check line before running anything. Mixing values from different rows is the mistake to avoid, and the check line is what catches it.
Then run a transcription model that returns per-word times. NVIDIA's own example is Parakeet TDT 0.6B v3, which covers 25 languages and carries a CC-BY-4.0 licence, meaning you credit it and otherwise get on with your life.
Then join the two lists on the midpoint of each word. The working version of that join is in the "Combine diarization with Offline ASR" section of NVIDIA's own writeup, not in the repository's integration guide. That guide covers something else: a tighter live pipeline where the two jobs run coupled together, which is a different tool for a different problem.
One setting decides everything, so get it right. The model runs at four different speeds, described by how much audio it waits for before answering: 30.4 seconds, 1.04 seconds, 0.64 seconds, and 0.32 seconds. The instinct is to reach for the fast one. Do not. For a file you already have, the slowest setting is both the most accurate and the fastest to chew through the whole recording, by a wide margin in NVIDIA's own figures. The short settings exist for live captioning, where you have no choice. If you are working from a recording, take the 30.4-second setting and forget the others exist.
Where it breaks
Eight speakers, hard stop. A roundtable with nine people, a crowded dinner scene, a press conference: speech gets dropped or handed to the wrong channel, and the model will not warn you. Someone asked about exactly this in the comments under NVIDIA's own post a day after it went up, naming press conferences and earnings calls, which tells you the limit bites in real use.
The labels are not names. You get speaker_0 and speaker_1. The model is explicit that these are anonymous channels, not identities. It cannot tell you that speaker_2 is Maria. You map channels to people yourself, once per recording, which is a two-minute job but a job.
Overlap is honest rather than smart. When two voices are active at the same instant, the join marks the word ambiguous instead of guessing. That is the right behaviour and it is also exactly the moment you wanted help. Words falling outside any detected speech come back unassigned. Expect to hand-fix the top of every panel discussion where everyone greets each other at once.
The full pipeline wants Linux and an NVIDIA card. The model card says Linux with an Ampere, Hopper or Blackwell card. If you are on a Mac, the Argmax route is your route, and it is a commercial SDK with its own pricing beyond the free test app.
The benchmark headline needs a pinch of salt, and NVIDIA supplies it themselves. The figure in every writeup is a first-place finish on VoiceArena's diarization benchmark, 14.72% error against 19.3% for the next system. NVIDIA's own post says those initial results may change as that benchmark completes its first full evaluation. A first-place finish on a benchmark in its launch week is a reason to test, not a reason to believe.
The part worth reading twice
Go back to the licence. It is the OpenMDW agreement, version 1.1, from the Linux Foundation, and it contains this: the agreement "does not impose any restrictions or obligations with respect to any use, modification, or sharing of any outputs generated by using the Model Materials."
Sit with that for a second, because it is not the norm. Plenty of open-weight releases will let you download the files and then attach strings to what you make. Here the strings are explicitly cut. The one condition that matters in practice is ordinary: if you redistribute the model itself, ship the licence and the notices with it. Using it on client work and delivering the result is simply not the licence's business.
For a small studio that is the whole calculation. The model is 107 megabytes, it is free, it runs on hardware you might already have, and nobody has a claim on the transcript that comes out.
I suspect the interesting thing about this release is not that it beat a benchmark. It is that speaker separation has stopped being a service you rent and become a file you keep, small enough to sit in a project folder next to the footage. Once a capability crosses that line it tends to show up inside the tools you already use within a year or so, and stop being something you think about at all.
Which raises the only question I actually care about here: when your editing suite ships this natively, will it be honest about the overlaps the way this model is, or will it pick a speaker without telling you and let you find out in the edit? I know which one I would rather debug.
If you run this on a multi-speaker recording with heavy cross-talk, tell me what fraction of words came back ambiguous. That number is the real review.
Medium metadata
Suggested kicker: Two models, one stopwatch, fifteen lines of code, and one setting that decides whether it works.