FervorCreative AI
Live Latest 13.09.26 · morning 63 tools tracked 100 workflows indexed 143 topics Hot: MiniMax H3, ComfyUI, LTX-2.5

Analysis September 13, 2026 workflow 9 min read

The release worth copying is the hyperparameter list, not the safetensors file, because it turns a missing control type from a lab problem into a ten-hour errand.

AnimaControlNetComfyUIsd-scriptsimage-gencreative-workflowslora-finetuninglicensing-provenance

Ten Hours on One GPU Buys You a ControlNet: The Anima Canny Recipe, Printed in Full

Someone trained an edge-control adapter for a model they did not build, published every setting, and made the procedure more valuable than the file.

Most adapter releases hand you a .safetensors and a preview image. You download it, it works or it does not, and if you wanted the same thing for a different control type or a different base model you would be starting from nothing.

This one, published September 13, hands you the settings. Dataset size and source. The exact Canny thresholds, and a note that they were held fixed rather than randomized. Which four blocks of the base model the control stack attaches to. Optimizer, betas, precision, timestep sampling, warmup steps, caption dropout. Peak VRAM. Wall-clock cost: about ten hours on one L40S.

That last number is the interesting one, because it is small. Ten GPU-hours is a Saturday and a rented card. Which means "the base model does not support the control type I need" has stopped being a thing you wait for someone else to fix.

What is actually new here, and what is not

Worth getting this straight, because the framing matters.

Anima is a Cosmos-Predict2-derived anime image model from circlestone-labs with over a million downloads and a large adapter ecosystem. It has had structure control for months. TaihoC published a VACE ControlNet training implementation and a depth adapter built with it on June 20. The community had already shipped LLLite adapters for regional prompting (June 1) and tile repair (June 16), and added inpainting in August.

So this is not the first Anima ControlNet. It is a new control type, and, as far as I can find, the first time somebody other than the tooling's author used that training implementation and wrote down what they did.

The difference between depth and Canny is a real one for anyone drawing. Depth control needs a depth pass, which usually means running an estimator over a reference image or building something in 3D. Canny control takes edges, which means you can trace a rough composition, scan a pencil sketch, or run an edge detector over a photo and get structural guidance out of the same afternoon. It is the cheapest way to say "put the character here, facing this way, with the building behind her."

The adapter itself is worth having. The recipe is worth copying.

The one decision that separates a working ControlNet from a nearly-working one

Buried in the training details, one line does more work than everything around it:

Canny Edge Maps: Generated using cv2.Canny with min_threshold=100 and max_threshold=200 (no random thresholds).

That parenthetical is the whole ballgame, and it goes against common practice.

Randomizing the Canny thresholds during training is standard advice. The reasoning is intuitive: vary the input so the model generalizes across edge maps of different densities, and it will handle whatever the user throws at it. Plenty of ControlNet training scripts default to it.

The cost is that you train a model to be tolerant of everything and precise about nothing. An edge map at threshold 50/150 is a thicket of noise, and one at 200/400 is a few isolated contours. A model trained across that whole range learns "these lines are a suggestion." A model trained at exactly 100/200 learns what a 100/200 edge map means, and follows it.

The mechanism, as I read it, runs like this. Train at fixed thresholds and then generate your inference-time edge maps with the same thresholds, and the model is reading a language it has seen ten thousand times. Get that wrong and you have handed it a dialect. Structure comes out approximately right, you blame the adapter, and the adapter was never the problem.

That is an argument rather than a measured result on this model, and I will say so again in the limitations. It is also, in my experience, a common and easy-to-miss reason a home-trained ControlNet underperforms a published one. Not the dataset. Not the learning rate. The preprocessing drifted between training and use, and nobody wrote down what it was at training time.

Which is exactly why publishing the thresholds is the useful act.

How four blocks steer a twenty-two block model

The architectural choice is the other thing worth understanding before you copy it.

A ControlNet is not a modification of the base model. It is a small parallel stack that reads your control image and injects hints into the base model's residual path at chosen points. The base model is frozen. Whatever it knows about drawing hands, rendering hair, and handling your prompt stays exactly as it was.

This adapter uses four control blocks, connected at base blocks 0, 7, 14 and 21. Spread evenly across the depth of the network, not clustered.

The reason that works maps onto something familiar. Early blocks in a diffusion transformer settle the coarse arrangement of the image, the where-things-are. Late blocks handle surface: texture, edges, small detail. A tap at block 0 says "the figure goes here." A tap at 21 says "and this contour is a real contour, not noise." Two taps in the middle keep the two from drifting apart while the sampler runs.

You could attach a control block to every layer. It would cost more memory, train slower, and mostly teach the adapter to fight the base model. Four taps is enough of a hand on the wheel.

Put this into practice

Here is what copying this actually involves. I am going to be honest about which steps are easy and which are the afternoon.

1. Get the tooling. Clone the feat/anima-vace-controlnet branch of TaihoC/sd-scripts-animaCN, which is a fork of kohya-ss sd-scripts. Ten minutes.

2. Build the dataset. This is the afternoon. The published adapter used about 15,000 captioned images at 1024×1024, from the public RicemanT/booru-essence-2026 set. If you are training a style or subject nobody has assembled, this is where your time goes and no recipe helps you.

3. Generate the edge maps, and write down your thresholds.

import cv2

MIN_T, MAX_T = 100, 200   # write these in your model card

edges = cv2.Canny(cv2.imread(path, cv2.IMREAD_GRAYSCALE), MIN_T, MAX_T)

Two lines, and the constants at the top are the part that matters six months from now. Put them in a config file that both your training script and your inference preprocessor read, so they cannot drift apart. Nothing else in this recipe is as cheap to get right or as expensive to get wrong.

4. Configure the training run. The published settings, in full:

control blocks     4, attached at base blocks 0, 7, 14, 21
batch size         16  (2 per GPU, gradient accumulation 8)
training steps     3000
learning rate      5e-5, cosine, 150 warmup steps
optimizer          AdamW_adv
  betas            (0.9, 0.99)
  use_atan2        True
  orthogonal_gradient   iterative
  state_precision  bf16_sr
  stochastic_rounding   True
timestep sampling  shift, discrete_flow_shift=3.0, sigmoid_scale=1.0
caption dropout    10%
precision          full bf16
resolution         1024x1024
gradient checkpointing   off

Two notes on the less obvious entries. stochastic_rounding with bf16 optimizer state is what makes full bf16 training survive 3000 steps without the small updates rounding to nothing, which is the classic way a low-precision run silently stops learning. And 10% caption dropout forces the adapter to steer on the edge map alone for one batch in ten, so the control signal does not become a passenger that only works when the prompt already describes the composition.

5. Rent the card. Peak VRAM is about 32 GB with gradient checkpointing off, so this needs a 40 GB-class GPU or larger. Third-party price trackers currently put an L40S around a dollar an hour on the cheaper clouds, which puts ten hours in the region of ten dollars, though rates move and you should check the provider directly rather than trust that figure. Turning gradient checkpointing on will fit it into less memory at the cost of a longer run.

6. Install the fork you will need at inference. The adapter card is explicit: it requires the fix/anima-vace-hardening branch of PineCookie/ComfyUI-Advanced-ControlNet, because TaihoC's fork currently fires a duplicate control hook. Do this before you spend the money, not after. There is no worse debugging session than one where the trained thing and the loader are both suspects.

7. Generate with matched preprocessing. Same cv2.Canny(image, 100, 200). Same resolution. If your inference edge maps look denser or sparser than your training ones, the model is guessing.

Honest limitations

The license travels, and it is not permissive. Anima ships under circlestone-labs-non-commercial-license, and the adapter inherits it. So does anything you train on Anima yourself. This is a portfolio-and-experiments recipe, not a client-work recipe, unless you have separately sorted out terms with circlestone-labs. Worth confirming before you spend a Saturday.

Thirty-two gigabytes is not a consumer number. A 24 GB card does not run this configuration as published. You can enable gradient checkpointing, cut the per-GPU batch to 1 and raise accumulation, or drop to 768×768, but each of those changes the run, and the ten-hour figure stops applying. Nobody has published the modified numbers.

One adapter, no ablations. There is no comparison against a randomized-threshold version of the same training run, no sweep over the number or placement of control blocks, and no evaluation metric beyond a preview image. The fixed-threshold argument I made above is mechanically sound and matches what I have seen, and it is still an argument rather than a measurement on this specific model.

The fork chain is fragile. You are on a feature branch of a fork of sd-scripts for training and a fix branch of a fork of Advanced ControlNet for inference, and both exist because upstream has not absorbed this work. Neither has meaningful star counts. That is normal for two-day-old ecosystem plumbing, and it also means an upstream change could strand you.

Three thousand steps at batch 16 is modest. That is roughly 48,000 image-views over a 15,000-image set, so a bit over three epochs. Enough to teach edge-following. Probably not enough to make it hold up across art styles far from Danbooru.

What this changes about how you plan

The thing I would sit with is the cost curve, not the adapter.

A ControlNet used to be something a lab or a well-known community trainer produced, and you waited. At ten GPU-hours and one published config, it is a line item. If you work in a style the public adapters handle badly, or you need a control type nobody bothered with (normal maps, palette conditioning, layout boxes, whatever your actual pipeline needs), the blocker is now assembling 15,000 captioned images, and that was always the real blocker.

So the question I would ask of your own setup is not "does a ControlNet exist for this." It is: what would you condition on, if conditioning cost ten dollars and a weekend? Most people have never had to answer that, because the answer never mattered.

If you train one, publish the thresholds. That is the part of this release that will still be useful when the weights are obsolete.

Medium metadata

Suggested kicker: Training your own adapters

Primary sources: