Why AI-Refined Faces Wobble in Video, and the ComfyUI Fix That Pins Them Back On
A small face in a wide AI shot comes out mushy. Re-drawing it sharp makes it slide around on the head. A new free ComfyUI pack explains why, measures it, and fixes most of it with a trick any compositor will recognize.
AI video has a size problem with faces. Put a person in a close-up and the face holds up. Pull back to a wide shot where the head is forty or sixty pixels tall and the same model draws a soft smear with eyes. Render at a higher resolution and it barely helps, because the problem is how much of the frame the head occupies, not how many pixels the frame has.
The community answer has been a face refine pass, borrowed from still-image tools. Track the face, crop it so it fills a frame, let the model re-draw it at that size, then shrink it and paste it back. The face comes back sharp.
Then you play it, and the face swims. The eyes and mouth seem to float on a head that is not moving with them. It looks like a bad tracking job, so people try to track better. That does not work, and a ComfyUI pack published this week explains why.
The wobble is not where you think
ComfyUI-H3-FaceRefine-Plus, by the GitHub user AnniCodex, is an add-on to Carasibana's ComfyUI-H3-FaceRefine, a face refiner for MiniMax H3 video. The base pack tracks the face and pastes the result back with the exact reverse of its crop, so the box itself is steady.
The Plus author's diagnosis: the wobble "happens inside H3." When the model re-draws the face, it puts the eyes, brows and mouth corners "1-2 px away from where they were, and the offset is different on every frame." Hair, ears and the outline of the head keep their original pixels. So the new features slide around inside an old head that is not moving with them. Better tracking cannot fix that, because the tracking was never wrong.
Once you see it that way, the fix is not a better AI. It is a compositing move.
How Flow Lock pins the face back on
The pack's answer is a node called H3 Face Flow Lock. It does what a compositor would do by hand with a stabilizer, except per pixel:
- It compares each refined frame with the original frame and measures how far every bit of the face moved. This measurement is called optical flow: a map of which way each patch of pixels shifted. It runs on slightly blurred gray frames, so it follows the face's structure and ignores the new skin texture.
- It only looks where the refine pass actually changed the picture.
- It smooths that map, so it follows the overall slip and never the new fine detail.
- It nudges the refined face back by that amount, frame by frame, and blends it in.
The author deliberately did not smooth across time, because the wobble jumps from frame to frame and averaging neighbors only removed half of it.
On the author's test shot (two people, faces about 206 pixels tall), adding Flow Lock after the pack's improved stitch cut the wobble from 0.73 to 0.19 percent of face height, and the face came out slightly sharper. Those are the author's own measurements, published in the README, on their own footage, but they come with the method, the settings and the code.
The color trap hiding under every AI video edit
The second fix in the pack matters even if you never refine a face.
Refined shots often come back darker and more magenta, with purple lips. Most people blame the AI. The author traced the main cause somewhere else: video decoding. Most video is tagged with a color standard called BT.709, a label that tells software how to turn the stored numbers into colors. The author found that ComfyUI's own video loader and VHS Load Video read that footage with the older BT.601 recipe, while VHS Video Combine saves proper BT.709. The frames the face pack itself loads carry the same shift. So a clip that goes in and out of ComfyUI with no processing at all already shifts color. On one shot the author measured lips noticeably darker and bluer after a plain load and save.
The fix is boring and important: load a clean reference copy of your source with VHS Load Video FFmpeg, which reads the color tags, and feed that copy to the pack's H3 Face Tone Match node. Tone Match then pulls the refined face back to the source's tone and color, and a lip pass corrects the lips separately because they are smaller than the tone correction's blur. On the author's shot, the lips' color error fell to within about one unit on each color channel with the lip fix on, small enough that most viewers will not see it.
If you have ever wondered why your ComfyUI exports look slightly off against the original in your editor, check your loader before you blame the model.
Put this into practice
You need a working ComfyUI with MiniMax H3 set up, and an NVIDIA card. The author ran everything on an RTX 5090.
1. Install both packs. Install Carasibana's ComfyUI-H3-FaceRefine first. Then:
cd ComfyUI/custom_nodes
git clone https://github.com/AnniCodex/ComfyUI-H3-FaceRefine-Plus
Restart ComfyUI. The new nodes appear under MiniMax H3 / Face Refine.
2. Run the example before your own footage. Copy the repo's assets/ folder into ComfyUI/input/h3plus_example/. It contains four invented characters, each with a wide shot, a face reference, a character sheet and prompts. Open face_refine_plus_one_shot_workflow.json, set the full path to mara/sample_shot.mp4 in the text node at the top left, and queue it. Seeing a known-good result first tells you whether a later problem is your setup or your shot.
3. Trim your shot to a length H3 accepts. H3 works on 22, 39, 56, 73, 90, 107, 124 or 141 frames (17 times a number, plus 5). Trim down to the nearest one. The author warns that a shot padded by more than a few frames gets a grid of noise on the faces in its final 17-frame group.
4. Pick the refine strength. The pack's strength node offers standard 0.15 (recommended) and strong 0.30. Standard keeps the face the model first drew. Strong moves it toward your reference pictures. Doubling the strength did not add detail in the author's tests.
5. Wire the finishing chain in this order. H3 Face Stitch Plus (replaces the stock stitch, with cleaner placement and lighting taken from the original frame), then H3 Face Flow Lock, then H3 Face Tone Match fed a copy of the source loaded through VHS Load Video FFmpeg. The example workflow already does this.
6. Judge it in motion, at 100 percent. Scrub the face frame by frame and then play it at speed. Wobble shows in playback, color shows in stills. Compare against the original shot side by side.
7. If the refined face looks like a cousin, try Method B. The author's own approach for long projects: refine one wide shot, turn the refined face into a new reference portrait, rebuild the character sheet from it, and re-render the clip from scratch with the same prompt and seed. Across the four example characters, likeness between the refined wide shot and the close-ups rose from 0.55 to 0.65 on the author's face-match score. That gain is measured against the rebuilt character, though: on one of the author's test shots, the Method B face matched the original reference less well than before. Do this at the start of a project, not mid-way, because it changes the face a little.
Where it breaks
It only works on MiniMax H3 footage. The nodes are built around H3's crop, re-draw and paste steps. They will not fix a face from another video model. And MiniMax's H3 community license, as quoted on the model cards built on H3, excludes the United States, the European Union, the United Kingdom and South Korea, so check where you are before you use the output commercially.
It is slow. On the small-face test, the official refine took 87 seconds on the RTX 5090 and the Plus versions took 239 to 256 seconds. Plan on roughly 2.5 to 3 times as long per shot.
Standard strength trades likeness for steadiness. At standard 0.15, the refined face matched the character's close-ups less closely than the official refine did on the author's small-face test. Strong fixes that but re-draws more of the face. You are choosing between faithful to the shot and faithful to the character.
The evidence is one person's tests. Every number here comes from the pack's author, mostly on one clip per case, on invented characters. That is far better than nothing, and the methods are written down, but your faces, lighting and motion will differ.
It is not a fix for a bad performance. If the model drew the wrong expression or a broken head turn, a refine pass makes a sharper version of the same mistake.
Some example footage is not yours to reuse. The author's test page marks its music-video clips as all rights reserved. Use the invented characters in assets/ to learn the workflow.
The fix was in the cut, not the model
The useful lesson here reaches past one ComfyUI pack. When generated video misbehaves, the reflex is to wait for a better model or reroll with a new seed. This author did something else: watched the failure closely, named its cause, measured it, and fixed it with tools a compositor has used for decades. Optical flow, frequency separation for skin, color management at the loader.
So the next time an AI face swims on your timeline, scrub it frame by frame before you reroll. Ask whether the head is moving or only the features. If it is only the features, you already know the fix, and you can pin them back on yourself.