FervorCreative AI
Live Latest 30.08.26 · morning 20 tools tracked 15 workflows indexed 48 topics Hot: ControlFoley, MiniMax H3, SANA-Video 2.0

Analysis August 30, 2026 workflow 13 min read

The commercial music generators still have no score underneath, so any MIDI they hand you is transcribed back out of audio they already rendered; a small set of open models emit the notes directly, and that is what lets you finish the music yourself.

text2midiMIDI-LLMMidiCapsMuseScoreSunomusic-genopen-weightscreative-workflowsai-editinglicensing-provenance

Text-to-MIDI Generation: How to Get a Score You Can Edit Instead of a Song You Cannot

A working pipeline from a text prompt to editable notes in a free notation editor, plus an honest account of how far the open models actually get.

A text-to-MIDI model is about 900 megabytes on disk and produces a file measured in kilobytes. Inside that file there is no sound at all. There is a list of integers: which key, how hard, when it starts, how long it lasts, on which of sixteen channels, at what tempo. Every one of those numbers is a number you can change.

That is a different product from what the music generators sell, and the difference is easy to state wrong. Suno's Studio does have a MIDI feature, and has since October 2025, when the v1 Studio shipped a "Get MIDI" command. Its own help page describes exactly what that command does: "Studio will analyze the audio and generate a MIDI file representing the melodic or rhythmic content." Ten credits, Premier tier only. Mureka sells MIDI export too, through the same shape of pipeline, returning it alongside stems.

So the notes come out of the audio, after the fact, by transcription. There is no score underneath. Studio 2.0, which shipped on August 13 this year, actually makes the point cleaner: it added a piano roll, Musical Typing, and hardware MIDI input, so you can now play symbolic notes into Suno, and its documented export formats are 32-bit WAV and MP3. Real MIDI going in, rendered audio coming out.

A symbolic model inverts that. It never renders anything. It emits the score first, and the sound is a decision you make afterward.

Why anyone should care about the difference

If you want a finished track for a video, none of this matters. Generate audio, take the audio, move on.

It matters when you want to change one thing. Transcription is lossy in a way that compounds. A piano stem with sustain pedal produces overlapping decays, a transcriber guesses at note-off times, and what comes back is approximately right and rhythmically smeared. Spotify's basic-pitch is Apache-2.0, free, and genuinely good at this, and its own README still says it works best on one instrument at a time.

A symbolic model has no such step. What comes out is what the model decided, at full precision, because the notes were never audio. Move a melody up a third by adding 4 to a set of integers. Halve the tempo without touching pitch, because tempo is a separate file-level event that never interacts with the notes. Change one Program Change byte and the piano part becomes a cello part. Print the thing as engraved sheet music in a free application.

Where I land: for anything you intend to arrange, orchestrate, hand to a player, or turn into notation, symbolic generation is worth its considerably rougher output. For anything you intend to ship as-is, it is not close, and pretending otherwise wastes your afternoon.

How these models actually work

A MIDI file is already a sequence of discrete events, so you can hand it to a language model with almost no adaptation. The adaptation that is needed is the tokenizer, and the two models worth your time picked different ones.

REMI is the most common choice in this lineage, and MidiTok implements it. REMI represents each note as a run of Pitch, Velocity and Duration tokens, and represents time with Bar and Position tokens, where Bar says a new measure has started and Position says where inside it you are. Optional families add Tempo, Time Signature, Chord, Rest and Program tokens. The tokenizer config that ships with a model is what makes its vocabulary reproducible; without the exact one it was trained with, generated token strings will not decode back into the MIDI anyone intended.

text2midi puts a frozen FLAN-T5 encoder on the text side and an 18-layer autoregressive decoder with 8 attention heads on the MIDI side, cross-attending into the encoder's hidden states. 272 million parameters total, 159 million of them trainable, with a 2,048-token context. It uses REMI+, the multi-track variant that adds program and time-signature tokens, and its outputs typically run 30 to 40 seconds.

MIDI-LLM takes the other road twice over. It starts from Llama 3.2 1B and expands the embedding table to 183,286 tokens so the MIDI vocabulary lives alongside the text vocabulary in one model, then full fine-tunes rather than using an adapter. That preserves the parameter structure, which is why it runs under stock vLLM with no custom serving code. And it explicitly rejects REMI in favor of the Anticipatory Music Transformer's arrival-time tokenization, on the grounds that REMI needs beat-synchronized data and arrival-time tokens do not.

Both use MidiCaps captions, which is 168,385 MIDI files from the Lakh dataset paired with text. The captions were not written by people. They were generated by Claude 3 Opus from musical features extracted programmatically, which the MidiCaps authors state plainly in their ISMIR 2024 paper, and the model never heard any audio. Worth holding in mind when you wonder why prompts describing production or timbre get ignored: the training captions describe key, tempo, instrumentation and mood, because that is what the feature extractor could see.

Put this into practice

Two paths. Take the first if you want it to work, the second if you need permissive licensing end to end.

Path A: the one that runs today

MIDI-LLM is the most reliably reproducible option this week. Its repository was pushed in late July, it ships a script that takes real arguments, and it outputs both .mid files and synthesized previews. Its README asks for a GPU with 16GB or more of VRAM and CUDA 12.x, and the generation script hardcodes a CUDA device, so this path is not a Mac path.

1. Grab a soundfont first. Synthesis is on by default and the script exits if it cannot find ./soundfonts/FluidR3_GM/FluidR3_GM.sf2. Put one there, or pass --no-synthesize and skip straight to the MIDI.

2. Generate.

python generate_transformers.py \
  --model slseanwu/MIDI-LLM_Llama-3.2-1B \
  --prompt "melancholic solo piano, 72 bpm, D minor, sparse and rubato"

That gives you four takes by default. --n_outputs, --temperature, --top_p and --max_tokens are all there if you want them.

Write prompts like that one. Every model here responds to explicit musical parameters, tempo in BPM, a key, a mood, an instrument, and responds badly to a single adjective. This is not prompt mysticism. It is a direct consequence of what the MidiCaps captions contain.

3. Open it in MuseScore Studio 4.7.4. Free, GPL-3.0, released this July, on Windows, macOS and Linux. File, Open, pick the .mid. You now have staff notation you can edit note by note, transpose, re-voice, and export as PDF or MusicXML. One thing to know before you go hunting: the detailed MIDI import panel with quantization grids and tuplet search belongs to MuseScore 3. MuseScore 4's handbook documents a single Preferences setting for the shortest note value on import, so expect to clean up rhythm by hand rather than by dialog.

4. Make it sound like something. MuseScore's bundled MS Basic sounds play it immediately, which is enough to judge whether the notes are worth keeping. When they are, render with FluidSynth 2.6.0 (LGPL-2.1) against a real soundfont. FluidR3 is MIT, in its author's own words. GeneralUser GS is 30.7 MB with 261 instrument presets and 13 drum kits under a bespoke permissive license that says you may use it "without restriction for your own music creation, private or commercial."

This is the step people skip and then blame the model for. General MIDI is a naming convention, not a sound. The MIDI Association's own General MIDI Level 1 page says program numbers map to names like "Electric Piano" and "Oboe" but "the acoustic characteristics of the each sound are not defined." Your output sounds exactly as good as whatever you route it to.

5. Take it into a DAW when you want to arrange. LMMS is genuinely free under GPLv2 on all three platforms and loads both MIDI and SF2. GarageBand is free on a Mac. Be careful with the usual recommendations: REAPER is $60 for a discounted license, which covers personal use, educational and non-profit work, or a business under $20,000 yearly gross revenue, and $225 otherwise, after a 60-day full-function evaluation. Ardour's source is free under GPLv2 but its demo binary goes silent after ten minutes.

Path B: permissive licensing, with a documented command that cannot work

text2midi is the credentialed option. AAAI 2025 paper, MIT code, Apache-2.0 weights, a published listening study. It is also the one that will waste your evening if you follow its instructions.

The README tells you to run python model/transformer_model.py --caption "your description". Open that file and line 1410 loads its weights from /root/test/text2midi/output_new/epoch_30/pytorch_model.bin, an absolute path on the authors' own machine. It downloads nothing. It reads its tokenizer from a relative ../artifacts, and it writes to a hardcoded filename, output_christmas_2.mid, regardless of anything you pass it. --caption is the only argument the parser accepts. A second generation function hardcodes three more /root/ paths.

The official Hugging Face Space is also down, and not sleeping: it throws ImportError: cannot import name 'HfFolder' because it pins gradio 4.41 against a huggingface_hub that removed that class in version 1.0.0 last October. The repository has not been pushed since February 2025, so nobody is going to fix it.

The working path is the README's Quickstart Python snippet, which does the right thing:

from huggingface_hub import hf_hub_download

ckpt = hf_hub_download("amaai-lab/text2midi", "pytorch_model.bin")   # ~902 MB
vocab = hf_hub_download("amaai-lab/text2midi", "vocab_remi.pkl")     # 27 KB

Two details that save time. The Hugging Face model card's training command passes --encoder_model="google/flan-t5-large" to a train.py that is not in the repository, while every flan-t5 reference in the actual code loads google/flan-t5-base. Use base. And the Hugging Face repo ships its own 155 MB soundfont.sf2, so step four above is already solved for you.

Pin a Python 3.10 environment for this one. The requirements file specifies torch==2.5.1, numpy==2.0.2 and miditok==3.0.3, all roughly twenty months old, and they will fight a current interpreter. No inference VRAM figure is published for text2midi anywhere. I looked. Do not trust a number you see quoted for it.

Honest limitations

Chords are the weak point, and the authors say so. In text2midi's own listening study, chord matching scored 2.50 out of 7, the lowest of its seven questions. Human ground-truth MIDI from the same test scored 3.20, so the measure is harsh, but the gap is real and it is the first thing you will hear. Melodies and arpeggios come out plausible. Voice leading does not.

The listening study is eleven people. Eleven. A reasonable pilot and a poor basis for a strong claim in either direction.

The published metrics have moved, and the explanation for why does not hold. The first arXiv version of the text2midi paper reported a Correct Key score of 7.69% against MuseCoco's 13.70%, a loss, in a paper whose prose says it "consistently surpasses MuseCoco on all objective metrics." The current version reports 33.60% and the contradiction disappears. The GitHub README attributes the change to the paper's numbers being computed "on a small subset of MidiCaps test set," but both versions of Table 2 carry a caption saying the results are averaged over all captions from that test set. Somebody's description of their own evaluation is wrong, and it is not obvious which.

Length is capped by the context window. Roughly 30 to 40 seconds for text2midi. You are generating material, not arrangements.

Instrumentation requests get ignored. The paper documents a requested church organ that never appeared, and attributes it to captions describing whole songs while only excerpts were used in training.

The licensing is not as clean as the badges suggest. MidiCaps is CC-BY-SA-4.0, a share-alike license, and every model here trains on it while releasing weights under more permissive terms. That is the authors' position, not a settled question. MIDI-LLM's weights carry Meta's Llama 3.2 Community license, which has terms worth reading even if none of them will bind you. And this week's newest entry, SeaOtter 0.5 Beta, shows how the chain goes wrong: it wears an Apache-2.0 badge inherited from Unsloth's quantized mirror of Qwen2.5-3B-Instruct, but the original Qwen/Qwen2.5-3B-Instruct declares its license as other, named qwen-research. Not Apache. Check the original, always, not the relabel.

Two more things about SeaOtter, since it is what put this topic on my list. Its inference snippet loads a repository ID that does not exist on Hugging Face, and the tokenizer config the card tells you to read from the repo root actually sits in a subdirectory, so downloading the adapter folder does not get you one. It trained for exactly one epoch, and its card is refreshingly honest about the result: "After 1 epoch, the model mainly writes single note lines and rolling arpeggio patterns. It does not reliably play 3 or 4 note stacked chords at the same position yet." Read that as a note to the category rather than a knock on one hobbyist.

And do not claim audio is uneditable. Celemony's Direct Note Access identifies and edits individual notes inside polyphonic audio, and has for years. It is paid, it exists only in Melodyne's editor and studio tiers, it works best on isolated tracks, and large moves smear because it is resynthesis. The defensible version of the argument is the one this article is built on: MIDI is the only one of the two that hands you the composition itself, in a form you can re-orchestrate and print for free.

Where I would push on this

The category is older than it looks. MuseCoco did text-conditioned symbolic generation in 2023. text2midi shipped in December 2024 from the same lab that built the caption dataset everyone trains on. MIDI-LLM landed at ISMIR 2026 and beats it convincingly on the numbers both report, 0.173 against 0.818 on FAD and 22.1 against 18.7 on CLAP. None of this is new. It feels new because the audio models got so good that symbolic output started reading as a deliberate choice rather than a limitation.

What bothers me is the measuring. Chords are the loudest failure in this whole category. text2midi's listening study puts them last. SeaOtter's card names them as the thing one epoch could not buy. And MIDI-LLM, the newest and best of the three, reports no chord metric at all, only FAD and CLAP, both of which compare embeddings of whole clips. The field's best model does not measure the thing most obviously broken, which means nobody can tell you whether it improved.

My guess is that simultaneity is the problem, not the models. REMI serializes a four-note chord as a run of sequential tokens sharing a Position, and arrival-time tokenization stacks them at the same onset by convention rather than by structure. In both cases "these four notes are one gesture" is something the model has to infer from a pattern rather than something the representation states. Somebody is going to try a tokenization where simultaneity is native. I think that is the release worth waiting for, and I would bet on it landing before the audio models grow a score underneath.

In the meantime the pipeline works well enough for one specific job: generating raw material at four in the afternoon when you have a key and a tempo and no ideas. Generate twenty, throw away nineteen, keep the four bars that surprised you, write the rest yourself.

If you get a usable chord progression out of any of these, I would like to see the prompt. I have not managed it.

Medium metadata

  • Title: Text-to-MIDI Generation: How to Get a Score You Can Edit Instead of a Song You Cannot
  • Subtitle: A working pipeline from a text prompt to editable notes in a free notation editor, plus an honest account of how far the open models actually get.
  • Tags: MIDI, AI Music, Music Production, Open Source, Machine Learning
  • Suggested publication: Tkay Nation's Writes

Sources