Fish Audio Drama 3: How to Direct a Multi-Voice Audio Scene From a Script
Text-to-speech has always been good at one person reading. Ask it for two people talking and the seams show. Each line comes out as its own little recording, every voice starts fresh, and the back-and-forth that makes a scene feel alive (the interruption, the beat before an answer, the laugh that lands on the other person's line) has to be faked afterwards in an audio editor.
Fish Audio's Drama 3, released October 13, is built for the scene rather than the sentence. You write a script with several speakers, put stage directions in square brackets, place breaths and laughs exactly where they happen, and tag single words for emphasis or a whisper. The model generates the dialogue together, as one performance.
The company's own advice is the most useful sentence in the launch post: "Drama 3 does its best work on scripted lines you render once and keep." A second line on the same page explains how to work with it: "Takes vary, so render a few and keep the one that plays."
Put those together and you have a workflow. Write it like a radio play. Cast it. Record several takes. Keep the best. That is closer to directing actors than to typing into a box, and it is the part most launch coverage skipped.
What changed
Most voice tools give you a slider for "emotion" or a dropdown of moods. Drama 3 moves that control into the text itself, in four forms.
Stage directions go in square brackets before a line: [defensive, slightly uneasy tone] I was home. A direction holds until the next one, and you can put several in one line to shift the delivery partway through. You can write the direction in a different language from the dialogue, which matters if your script is in Spanish but you think in English.
Placed sounds also go in brackets, at the exact point they should happen: [shaky breath], [small laugh], [pause].
Word tags change only the words they wrap. <stress>You</stress> were there that night. The documented set includes whisper, emphasis, soft, fast, slow, stress and forceful. In Fish's web editor you select words and pick from a Tags menu, so you never have to type the angle brackets.
Speakers get their own voices, and the whole exchange is generated in one go. Fish's model card says a single text can hold up to 16 speakers.
It covers 83 languages. For a working creator the takeaway is simple: the performance notes now live in the script, so when you rewrite a line the direction travels with it.
Why "render a few" is the real instruction
Fish says it tuned Drama 3 with one goal: a model of which take people preferred. That is how you would train an actor for feel rather than precision, and it explains the behaviour you will hear. Two runs of the same line will not sound the same. Usually both are usable. Sometimes one is far better.
Treat that as a feature and budget for it. If you plan your session around one render per line, you will be disappointed and blame the model. If you plan for three or four takes per line and pick, the way an engineer comps a vocal, you will get results that sound directed.
That also changes the maths. At Fish's launch price for the API ($10 per million characters until November 12, then the list price of $50), a 1,000-word scene is roughly 6,000 characters. Four full takes is about 24,000 characters, which costs pennies at either price. The real cost is your listening time, so keep scenes short enough to audition properly.
Put this into practice: a two-voice scene in one evening
1. Write the scene for the ear. Keep it under two minutes for your first try. Short lines, clear turns, one moment that needs a specific delivery. Read it aloud yourself once. If you stumble, the model will too.
2. Cast your voices. In the fish.audio web app you assign a voice to each speaker in the Text to Speech editor. You can pick from the voice library, or use Fish's separate voice cloning feature. If you clone, use your own voice or one you have written permission for. Fish's licence bans cloning a real person's voice without consent, and that is the line you want to be on anyway.
3. Add direction sparingly. Start with a direction only where the meaning would be ambiguous without it. A direction on every line flattens everything. One in four is a good start.
4. Place the sounds. Breaths, laughs and pauses go exactly where they belong in the text. A [pause] before an answer often does more than any mood instruction.
5. Tag single words last. Pick the one word per line that carries the meaning and tag it. <stress> and <whisper> are the two I would learn first.
6. Render four takes. Generate the full scene four times without changing anything. Listen to all four start to finish before you touch the script. Note which take wins each exchange.
7. Fix the script, not the take. If one line misses in all four takes, the problem is usually the writing or the direction. Change the bracket text, render again.
8. Assemble in your editor. Pull the winning take into your audio editor or video timeline. If one exchange is better in take two and the rest in take three, cut between them at a pause, the same way you would edit two takes from a voice actor.
If you work in ComfyUI, the same model is available as a partner node. Set the model to drama-3, connect up to five voices, and mark each line with its speaker tag, @Voice1, @Voice2 and so on. The ComfyUI documentation for the node shows how the tags sit alongside bracketed directions, so copy its examples for the exact order.
ComfyUI checks that every connected voice appears in the text at least once and stops with an error if one is missing, which is a useful guard against a dropped character. It bills per use through your ComfyUI account (about $14.30 per million bytes of text at launch pricing, rising to $35.75 from November 13).
Can you run it yourself?
Yes, with a large machine. Drama 3 is released with open weights, which means you can download the model and run it on your own hardware with no per-character bill. The full version is big. Fish's speed figures come from an NVIDIA H200, a data-center card.
The community moved fast. Within a day there was a smaller version Fish now hosts on its own account, about 22 GB to download, tested only on 80 GB-class cards. There is also an unofficial Mac version that needs about 26 GB of memory while it speaks and runs at about twice real time on an M3 Max, according to its page. That page is blunt that "Fish Audio did not release or endorse it," and it installs from a development branch of its audio library rather than a finished release. If you have a well-specced Mac and a free evening, it is worth trying. For client work, the hosted app is the safer path today.
Honest limitations
Takes vary. That is the trade for natural delivery. If you need the identical read every time, for example for a product name repeated across 40 ads, test carefully before you commit.
The best editing tools are not open yet. Fish has shown a way to regenerate a single changed word inside a finished take, and a way to ask for a take "softer" than the last one. Both are invite-only testing for now. Until they open, a script change means a new take.
The licence has a ceiling. Drama 3 uses the Fish Audio Community License. Research and non-commercial use are free. Commercial use is free only if you and your affiliates earn under US$500,000 a year. Above that, or if you want to offer it as a voice service to others, you need a separate deal with Fish. Small studios and freelancers are fine. An agency should read the terms first.
Consent is on you. Cloning a voice with Drama 3 takes a reference clip and its transcript. The licence prohibits cloning a real person without consent, but nothing technical stops a careless user. Keep written permission for every voice you clone.
The quality claims are Fish's own. The launch post reports Drama 3 at 93.2 percent on an independent pronunciation benchmark, against 88.1 percent for Fish's previous model. That is Fish reporting a third-party test. Your own script, with your own character names and jargon, is the test that counts.
The small builds cut corners. The smaller version's own page warns that takes heavy with tags are occasionally cut short. If you run it locally, listen to the end of every take.
The script is the session now
For years the work of AI voice was voice-hunting: scrolling libraries, tweaking sliders, hoping the next render sounded less like a car-park announcement. Drama 3 moves the work back to where audio drama always lived, in the script. The direction, the timing and the emphasis are all on the page, and the page is something you already know how to write.
So write one. Take a scene you have always wanted to hear, two characters and a problem, under two minutes. Mark it up the way you would for an actor, render it four times, and listen to which take plays. You will learn more about the tool in that hour than from any demo reel.
Medium metadata
Title: Fish Audio Drama 3: How to Direct a Multi-Voice Audio Scene From a Script
Subtitle: The new open-weight voice model puts speakers, stage directions, placed sounds and single-word tags in the text itself, and its makers' own advice (render a few takes, keep the one that plays) is the workflow.
Tags: Audio Production, Artificial Intelligence, Text To Speech, Podcasting, Audio Drama
Estimated read time: 8 minutes