Your Picture Model Is Not What Fills Your Graphics Card. The Prompt Reader Is.
Two people measured the same thing on the same weekend, one on images and one on video, and found the compression everyone does has been pointed at the wrong file. Here is how to check your own setup and what to swap.
Every model card leads with the same number: how big the file is that draws the picture. Every forum argument about running things locally is an argument about which shrunken version of that file looks least damaged. It is the whole vocabulary of local generation.
Over the weekend two people, working separately, one on an image pipeline and one on video, sat down and measured where the memory in a running generation actually goes. Neither seems to have known about the other. They published within hours of each other and reported the same result.
The file that draws the picture is not the biggest thing on your card. The file that reads your prompt is.
The image measurement put it at roughly 16 GB out of a 21.5 GB total. The video measurement was worse: 22.7 GB for the prompt reader alone, before a single frame had been drawn, more than the picture model and the decoder put together.
That is not a small correction. It means a year of community compression work, all of it real engineering, has been aimed at a file that is a minority of the problem.
Why nobody noticed
Because the prompt reader does not feel like a model. It feels like plumbing.
Here is what happens when you run a local image or video generation. Your text goes into one model, which turns it into a set of numbers describing what you asked for. Those numbers go into a second model, which starts with visual noise and refines it toward something matching the description, running the same operation over and over, typically forty passes for an image. Then a third, small piece turns the result into pixels.
Three models. Everyone thinks about the middle one, because that is the one with the name and the sample gallery and the personality. The first one gets treated as a text box with extra steps.
But the modern ones are not small. The image model in this weekend's case uses a vision-language model of about 8 billion parameters to read prompts, because it has to understand reference photographs as well as words. The video one uses a large multilingual reader that lands at 11.4 GB on disk. And in some setups that 11.4 GB balloons to 22.7 GB in memory, because the loading code converts it to a heavier number format on the way in without telling you.
So the mental model is wrong twice. The reader is big, and it is sometimes bigger in practice than its file size suggests.
The insight that makes this easy
Here is the part that turns a curiosity into a workflow.
The prompt reader runs once. It reads your prompt at the start, produces its output, and is done. Some setups release it from memory immediately afterwards.
The picture model runs forty times. Every refinement pass goes through it again, and each pass builds on the one before, so any error you introduce by compressing it compounds across all forty.
Compression works by storing each number in the model less precisely. Do that to the picture model and you nudge every one of forty steps slightly off course, and the drift accumulates. Do it to the prompt reader and you get a slightly less precise reading of your prompt, once, at the start, and then nothing else touches it.
Same technique. Completely different consequences, because of where in the chain it sits.
And the numbers back this up. The video write-up measured how close the compressed reader's output stayed to the original on a scale where 1.0 is identical. At eight bits: 0.995. The half-precision version everyone already runs without complaint: 0.997. Effectively the same reading of your prompt, in half the space.
The four-bit line you do not cross
This is the part I most want people to take away, because the temptation is obvious and the failure is not.
If eight bits is nearly free, why not four? The file drops to 3.2 GB. It loads faster. Everything about it looks better on paper.
The author tried it, measured it, and deliberately did not ship it. At four bits the similarity score falls to 0.911, which works out to a 44% average error in how your prompt gets read.
Their sentence about it is the best line published on compression in a while: "a 44% mean error in the embedding is a different prompt, not a cheaper one."
That is the whole thing. A four-bit reader does not give you a degraded version of your image. It gives you a clean image of something you did not ask for. And because the output is coherent and well-made, you will not recognise the failure as a failure. You will assume the model misunderstood you and rewrite your prompt, which will also not work, for the same invisible reason.
Eight bits. Not four. The line is real and somebody has already found it for you.
How to check and fix your own setup
Step 1: Find out what your pipeline is loading. In ComfyUI, look at the file names in your prompt-reader slot and check their sizes on disk. Otherwise just sort your model folder by size. You are looking for one file that is clearly the largest, with a name containing "t5", "clip", "encoder", or a language-model name.
Step 2: Compare that against the picture model. If the reader is bigger, or even close, you have found your problem. Most people are surprised here, because the picture model is the one they have been thinking about.
Step 3: Watch what actually loads. File size on disk and memory used are different numbers, and this is where the 22.7 GB came from. Open your system's activity monitor, or run nvidia-smi in a terminal on a Windows or Linux machine with an NVIDIA card, while a generation is starting. If memory use jumps to roughly double the reader's file size during the first few seconds, your loader is converting it to a heavier format. That is a two-times saving available for free.
Step 4: Swap in the eight-bit reader. For the image case, the ComfyUI maintainers already ship the alternatives: their Qwen-Image-2.1 bundle includes the picture model in two precisions and the prompt reader in three, including a four-bit one. You want the middle option, not the smallest. For video on Apple Silicon, the FastWan2.2 package is the reference implementation, with the compressed reader in t5_encoder_q8.safetensors.
Step 5: Expect the file name to look odd, and leave it alone. In that video package the compressed reader is named t5_encoder_q8 rather than t5_encoder. That is deliberate. In the author's words, a loader that does not expect compressed weights "fails immediately on a missing file rather than cryptically on mismatched shapes." It is a small kindness and it tells you something about how carefully the thing was put together.
Step 6: Generate the same prompt twice and compare. Same prompt, same seed, once before and once after. If the images are recognisably the same picture with minor differences, the swap worked. If you get something clearly different, you have been handed a four-bit reader by mistake. Go back and check.
Realistic effort: twenty minutes if you already have the alternative file, an hour if you have to download it. The saving on the video pipeline was 11.4 GB down to 6.0 GB on disk and 22.7 GB down to 6.0 GB in memory. That is the difference between a model running on your card and not running on it.
Where this does not help
It is not a speed fix. The reader runs once. Making it faster shortens a step that was already about a second. Your generation time is dominated by the forty passes through the picture model and, in video, by turning the result into frames. In the measured video case, a five-second clip took 311 seconds, of which the reader was 1.9. Shrinking it buys you memory, not time.
Your setup may not offer the choice. Some packaged tools bundle everything and give you no file-level control. If you cannot pick which reader loads, this is a feature request, not a workflow.
It does not help if the reader is already released. Some well-built pipelines load the reader, use it, free it, and then load the picture model, so the peak is one component at a time rather than all of them at once. That is exactly what the Apple Silicon package does, which is why its peak memory stays near its largest single part rather than its 18.8 GB total. If yours already does that, your ceiling is set by the biggest single piece, and shrinking the reader only helps if the reader is that piece.
The measurements are from two setups, not from yours. One image pipeline on NVIDIA cards, one video pipeline on an Apple M2 Max. The ratios will differ on other combinations. Step 3 above exists because you should measure rather than assume, including assuming this article is right about your machine.
Nothing here fixes licensing. The image model these measurements came from is research-and-evaluation only. Making it fit on your card does not make the output yours to sell.
The thing worth arguing about
What interests me most is not the fix. It is that the answer was sitting in plain sight in a file listing and it took two unconnected people measuring on the same weekend for anyone to say it out loud.
The ComfyUI maintainers, who have to make this work for everyone on every card, ship three precisions of the prompt reader and two of the picture model. Read that ratio again. The people closest to the problem have been solving it for months in the quiet way, by putting the right files in the folder, and the conversation around them has carried on being about something else entirely.
I think that happens because the picture model is the one with a name. Nobody posts a sample gallery from a text encoder.
So the question I would put to anyone running this stuff locally: what else in your pipeline have you never measured because it does not have a name? Go and look at your model folder, sorted by size. I would bet the top entry surprises at least half of you.
If you run the before-and-after and get a different result than the numbers here, post it. Two measurements on two machines is a finding. It is not yet a rule.
Medium metadata
Suggested tags: Machine Learning, AI Art, ComfyUI, Local AI, Stable Diffusion
Subtitle: Two people measured the same thing on the same weekend, one on images and one on video, and found the compression everyone does has been pointed at the wrong file. Here is how to check your own setup and what to swap.
Primary sources: FastWan2.2-TI2V-5B-MLX-q8 measurements · ModelsLab Qwen-Image-2.1 four-bit build · ComfyUI model bundle · Qwen-Image-2.1 architecture notes