Someone Finally Tested Six Ways to Train an Image Add-On. Five of Them Load Nothing.
A head-to-head on FLUX.2 Klein 4B, the two-render check it hands you, and why one old format keeps winning for a reason the scores never show.
Train an add-on six different ways on the same image model, with the same photos and the same budget, and you would expect the results to separate by a few percent. What went up on Hugging Face yesterday is stranger than that.
Five of the six trained files, loaded the way every tutorial tells you to load them, give you back images from the plain model. The add-on is never read. Nothing raises an error. There is a single line in the log about missing keys, and it scrolls past while you are already looking at the picture, deciding whether the training took.
I have watched people conclude their training failed when the file never got as far as being used. That is the finding worth your Sunday.
What was actually tested
A developer publishing under the name dptlab took FLUX.2 Klein 4B, the small open image model from Black Forest Labs that has been on people's disks since January and has passed 400,000 downloads, and ran the same job six ways.
The job is the one most people actually want from a trained add-on: show the model a few photos of one specific object, then get that object back in places it has never been. A particular pair of shoes on a beach. A client's product on a shelf you photographed separately. The training set was SynCD, which shoots each object under different lighting, backgrounds and poses, so the setting is already separated from the identity before training starts.
Same photos, same 500 training steps, same size, same random seed. The only thing that changed was the file format the finished add-on gets saved in. Six of them: LoRA, DoRA, LoHa, LoKr, OFT and BOFT. Every result is Apache 2.0, and the code behind it is MIT.
Here is what came back, scored on six prompts describing scenes none of the training photos showed:
| Format | Followed the prompt | Matched the object | Time per image |
|---|---|---|---|
| LoRA | 0.9646 | 0.4320 | 1.56 s |
| DoRA | 0.9572 | 0.4424 | 2.29 s |
| LoHa | 0.9710 | 0.2907 | 2.02 s |
| LoKr | 0.9502 | 0.2946 | 2.10 s |
| OFT | 0.9176 | 0.2750 | 1.60 s |
| BOFT | 0.8727 | 0.2680 | 2.44 s |
The two score columns fight each other, and the model cards say so plainly. A file that learned nothing at all scores beautifully on the first column, because a model you never touched follows prompts perfectly well. A file that memorized your training shots scores well on the second and ignores whatever you asked for. A format only wins if it moves both.
Two do. LoRA and DoRA are the only files that hold the object and still take direction. DoRA edges it on object fidelity and charges 47 percent more time for every image you render, which on a long batch is an hour you are paying for a difference you may not see.
The other four cluster near the "learned nothing" end, and the pitch each one was making makes that worse. LoKr's selling point is the smallest files of the six, and it does train roughly one fifteenth as much as LoRA. BOFT's is a denser rearrangement of the model on fewer parameters. LoHa trains exactly as many as LoRA and claims to pack more into each one. All three promises hold arithmetically and none of them survived contact with the actual job.
The part that is not about scores
Read the usage instructions on five of those six pages and you find the same warning, in the same words. This format is not something the standard loader can read. Try it and it logs "no LoRA keys found" and serves you the base model.
That is a failure mode with no symptom. You get an image. It is a competent image. It is simply not the image your add-on would have made.
The mechanism is boring, which is why nobody talks about it. The loader everyone uses scans a trained file for a particular naming scheme inside it. LoRA files use that scheme. The other five save their work under different names. The loader looks, finds nothing it recognizes, writes one line, and carries on with the untouched model.
It gets worse on the way out the door. The author tested two of the serving programs people run when they want a model behind a web endpoint, and reports that neither loads a non-LoRA add-on: both expect either plain model weights or a LoRA-shaped difference, and anything else either fails to load or, again, loads nothing and serves the base. So a file in one of those five formats is not merely awkward to test locally, it has to be folded permanently into the model weights before it can be served at all. There is a script that does the folding. What you get afterward is a complete copy of the model, several gigabytes, one per object you trained.
That is the finding the score table cannot show you, and it is the one I would act on. LoRA has an advantage that has nothing to do with quality: everything reads it. The local tools read it, the hosted services read it, and it stays a small file you swap in and out per job. Fifty-five megabytes for the winning file in this test. Every other format has to stop being an add-on before anyone can use it.
When two approaches tie on picture quality, that asymmetry decides which one you would actually ship.
Put this into practice
1. Run the two-render check on every add-on you own
This takes two renders and about ninety seconds.
Load your model, load the add-on, render one image and write down the seed. Then set the add-on strength to zero and render the exact same prompt at the exact same seed. If the two pictures are identical, pixel for pixel, the add-on was never loaded. Anything real, even at tiny strength, moves something.
Do this on the trained files you downloaded from strangers too, not only your own. It is the cheapest audit in this hobby.
2. Check the format before you blame the training
Open the file's page and look at its tags, or open the folder and read the config file next to the weights. If it says anything other than LoRA, the two-render check above is not a real test, because the loader will refuse the file either way and you will conclude the add-on is doing nothing when the problem is upstream of that.
3. Fold it in, if the format is not LoRA
The repository ships scripts/merge_and_export.py, which folds any of the six into the model weights and writes an ordinary model folder that every tool can open:
python flux2-klein-peft/scripts/merge_and_export.py \
--checkpoint runs/klein-peft/oft/subject-0/ckpt/final \
--out exports/klein-oft-subject-0
Be clear about what you are buying. You now have a full model on disk for one object. That is fine for a single client deliverable and terrible as a way of life.
4. If you compress, compress after folding, never before
Compressing a model to make it fit a smaller card is normal practice now. The order matters and the failure is silent again. A trained add-on is a set of differences measured against the exact weights it was trained on. Compress those weights and you have moved them out from under the add-on, which is still pointing at where they used to be. Fold first, compress second, and the question disappears because there is no add-on left to confuse.
5. Try the base before you commit a project to it
The free demo runs Klein 4B in a browser with no install. It will not run the six-way comparison for you, but it tells you in five minutes whether the model underneath is worth training on for the work you do. Apache 2.0, so anything you make with it can go in a paid job without a permission slip.
Where this falls down
Six prompts. One object. One seed. One model.
That is a small study and the author says so on every page, which is the main reason I trust the rest of it. The published table is a mean over a handful of renders, and on three-photo subjects the spread between runs is wide enough that a gap this size could partly be noise. The winning file's own page carries an unresolved warning that some of its trained layers may not have saved correctly, and the author labels the numbers provisional in bold at the top.
The repository has a Status section that still says no training run has happened, which is plainly stale now that six trained files are published, but it tells you how young this work is. The project has zero stars on GitHub. Nobody has checked it.
None of that touches the load-failure finding, which is the durable part. That is a property of the file formats and the loader, not of this particular test, and you can confirm it yourself in two renders without trusting a single number in the table.
And this is one model. Klein 4B is a small, permissively licensed, few-step model, and the ranking may go differently on a big slow one. If your work lives somewhere else, run the sweep on your own base before you throw out a format.
What I would want to know next
The interesting question this leaves open is how much trained work is sitting on people's drives right now that has never once been loaded. Not failed training. Not bad data. Files that came out fine and have never been read by anything.
If you have a folder of add-ons you gave up on, the two-render check is ninety seconds each. I would like to know what you find, because my guess is that the number is not small, and that a fair share of the "training is hard, I could never get it to stick" posts are really a naming convention problem wearing a costume.
Medium metadata
- Title: Someone Finally Tested Six Ways to Train an Image Add-On. Five of Them Load Nothing.
- Subtitle: A head-to-head on FLUX.2 Klein 4B, the two-render check it hands you, and why one old format keeps winning for a reason the scores never show.
- Tags: AI Art, Machine Learning, Generative Ai Tools, Stable Diffusion, Creative Technology
- Meta description: A six-way test on FLUX.2 Klein 4B found that five of the six common add-on formats are unreadable by the standard loader and silently serve the plain model. Here is the ranking, the two-render check that catches it, and what to do if your file is not LoRA.