Your Music Model Trainer Saves a File Called best. Do Not Use It.
Teaching an open music model a style on one gaming GPU works, and the documentation for it is finally good. The part nobody tells you is that the file the trainer recommends is usually the burnt one, and why.
There is a training tool for the open music model YuE2 that runs inside ComfyUI, turns a folder of songs into a style you can prompt, and fits on a 24 GB graphics card. It works. People are shipping with it. And buried in its own documentation is a sentence that should stop you short:
"On our runs the rungs the metric called overfit were the ones that sounded most like the band."
The tool's authors are telling you, in their own README, that the number their trainer optimises does not track the thing you actually want. Then a user who trained four separate style families with it published a model card that reaches the same conclusion from the other direction, with the receipts. One of his four styles, the file labelled best by the trainer, "sounds burnt." He shipped step 200 of a 350-step run instead.
Two independent parties, same finding. That is rare enough in this corner of the world to be worth a full article.
Why this costs you more than it looks like it should
The failure here is silent, which is what makes it expensive.
A model that fails loudly is cheap. It crashes, it outputs noise, you know in thirty seconds. This one produces a complete song. Verses, chorus, an arrangement that makes sense. It just sounds slightly cooked, like a master that got pushed too hard, and if it is the only version you ever hear you have no reference for what it should have sounded like.
So you conclude the style did not take, or the base model is not good enough, or your dataset was too small. You train again with different settings and burn another evening of GPU time on a problem that was never in your dataset.
The fix costs nothing. It is a change of habit, not of technique. But you have to know to make it.
The two halves, and why the trainer watches the wrong one
YuE2 is built in two parts that do different jobs.
One part writes the song. It decides the tempo, the key, how long the verses run, where the chorus lands, what the vocal melody does. Call it the writer.
The other part turns that plan into audio. Timbre, texture, the grain of the voice, the weight of the low end. Call it the sound.
When you train a style, both halves learn at once, in one run, and come out as a single file. That is a good design. The trouble starts with how the trainer decides which saved file is the good one.
It watches the writer's score. And the writer's score keeps improving, more or less forever, because predicting the next bit of a song you have already seen a hundred times is a task a model gets monotonically better at. Meanwhile the sound half peaks early and then starts degrading, and it degrades in a way that reads as "more confident" to a loss function and "harsh" to an ear.
The numbers from the reggae training runs make this concrete. One style's sound quality bottomed out between steps 150 and 200, then got worse through step 350, which is where the writer's score said stop. Another style, trained with the sound half capped at 600 steps instead of 1000, bottomed at step 200 and drifted by a rounding error through the rest of the run, so its final file was fine. Same tool, same person, opposite outcomes, and the difference was how far past the sound half's peak the run was allowed to go.
The tool's own guidance is blunter than mine: "Keep the whole ladder and listen."
The other finding, which is about prompting
The same model card contains a second result that saves you the thing everyone tries first.
If you want a fast, rapid-fire vocal delivery, writing "rapid-fire," "double-time" or "fast flow" in your style prompt does not work. The author tested it. Those words mostly push the writer toward a shorter, differently flavoured arrangement rather than speeding up the singing.
What works is line length in the lyric. Write verse lines at fifteen to seventeen words instead of the usual seven or eight, keep the chorus at normal length so the contrast lands, and the delivery roughly doubles in words per minute. Same model, same prompt, same seed. Only the lyric changed.
Once you know it, it is obvious. The writer half reads the lyric and fits it to bars. Cram more words into a line and it has to deliver them faster. The prompt describes the record; the lyric describes the phrasing. Asking the prompt to change the phrasing is asking the wrong half of the model.
He names the limit too. Keep the words short, one and two syllables, because dense lines full of long words overrun their bars and come out garbled. His rule of thumb is roughly words divided by two for the seconds a line needs, and when a line will not fit, shorten it rather than fighting the pronunciation.
Put this into practice
Here is the whole loop. Budget an afternoon for setup and an overnight run for the training itself.
1. Install the pieces. You need ComfyUI at v0.36.0 or newer, which has native support for the music model. Clone the FS_Audio Suite into ComfyUI/custom_nodes/, install its requirements, and put the base model file yue2_3b_bf16.safetensors from Comfy-Org/YuE2 into models/checkpoints/. The suite ships a downloader node that fetches the training pieces for you; run it once.
2. Build the dataset. Convert to FLAC, trim anything over 320 seconds, add a fade at the cut. Next to each song file, put a text file with the style description and a second one with the lyrics. If your sources are MP3 rips, transcode them first with ffmpeg -err_detect ignore_err, because a damaged leading frame crashes the dataset builder with an unhelpful message about a missing header.
3. Write the captions in the order you will prompt in. This decides whether your prompts work later, and it is the step most people rush. One descriptive sentence per song, trigger word first, then language, genre, vocal, instruments, mood, tempo, production. Not a tag list. The reggae card is explicit that tag-list captions produced a model that only responds to tag-list prompts and writes strange arrangements. Measure the tempo rather than guessing it: librosa.beat.beat_track on each file, rounded, written near the end of the sentence.
4. Check the vocal gender by hand. If you use a model to draft your captions, it will label high male voices as female often enough that every batch needs a pass. A wrong label in the caption becomes a wrong voice when you generate. Five minutes of work that saves a whole run.
5. Format the lyrics as tagged blocks. [Verse], [Pre-Chorus], [Chorus], [Bridge], [Outro]. Four lines to a verse. Do not add an empty [Intro] tag, because the writer will happily compose an eighteen-bar instrumental opening without being asked and you will lose your second chorus to the length cap. Transcribe at high confidence or leave the line out; a mis-heard lyric teaches the model the wrong syllable count for that bar.
6. Turn off automatic melody transcription. The card is direct about this: letting the trainer transcribe melodies itself put the vocal melody into the instrument voice and the rests into the vocal voice, and the sound half's score started climbing right after step 200. Set score-first to 0 and skip the transcription.
7. Train, and save every rung. The trainer writes out a file at intervals as it goes. Keep all of them. This is the whole point of the article.
8. Listen to the ladder. Render the same prompt, same lyric, same seed through every saved file. Not a handful of prompts, one prompt, held constant. Listen for the sound going hard and brittle. The last file before that happens is your model. On the documented runs that landed somewhere around step 200 of 350, but the number is not the lesson. The habit is.
9. Set the length cap to 360. At 150 the writer still composes full-length intros and interludes and your song truncates before the second chorus. The cap does not tell the writer to write shorter, it just cuts the render.
Two dials worth knowing once you are generating. Pushing the sound half's strength from 1.0 to 1.5 makes the output darker and rougher without touching the writing. Pushing the writer's strength above 1.0 collapses the vocal, and the measured example is stark: at 1.25 the model wrote one sung bar against 424 rests. Leave the writer at 1.0. Push the sound half.
Where this breaks
You cannot sell what comes out. The base model is CC BY-NC 4.0, which is non-commercial, and every style file trained on it inherits that. This is for your own work, your own releases, your own experiments. Not for a client. Not for a library you licence. That single fact rules this workflow out for a lot of people and there is no way around it short of a different base model.
The hardware floor is real but reachable. The tool's docs say a full song fits a 24 GB card at about 15 GB peak, and you want 40 GB or more to go longer. A 24 GB card is a used 3090 or a 4090. A real purchase, though a great deal cheaper than the video side of this hobby.
Listening to a ladder takes time. Ten saved files at four minutes each is forty minutes of attentive listening, and attentive is the operative word, because you are not listening for something broken. You are listening for something slightly too confident.
The vocal stays in one language family per style, and some seeds just produce a plan with almost no singing in it. The fix for that is to change the seed, not to raise the writer's strength, which makes it worse.
Every documented result here is one person's work on one tool. The reggae card is unusually honest and unusually detailed, and it corroborates what the tool's authors say about which saved file to pick, which is why I trust it. It is still two sources, not twenty. Run your own ladder and see whether your ears agree.
The part I keep thinking about
There is a general version of this hiding in the specific one. Every tool that trains something for you will eventually hand you a number and call it quality. The number measures what is easy to measure, and in creative work that is almost never what you care about.
The music case is unusually clean because the mismatch is structural. Two halves, two curves, and the trainer watches one of them. But image style training has the same shape, and so does voice cloning, and so will whatever ships next month. Somewhere in the pipeline a loss function is standing in for your taste, and it is not very good at it.
Which makes the practical advice depressingly simple and hard to automate: keep every saved file, hold one prompt constant, and go listen. If you have run a ladder like this and found the metric and your ears agreeing, I would genuinely like to know, because I have not seen it yet.
Medium metadata
Recommended reading time: 8 minutes
Primary sources:
- becausereasons/yue2-mltnt-militant-reggae (Hugging Face createdAt 2026-09-17 13:11 UTC, CC BY-NC 4.0)
- ComfyUI-FS_Audio_Suite, Artist Trainer documentation
- m-a-p/YuE2-3B (createdAt 2026-09-09, CC BY-NC 4.0, 3.63B parameters)
- Comfy-Org/YuE2, ComfyUI repack