Analysis September 5, 2026 tool 11 min read

VDN-H3's headline is an eight-GPU datacenter number, but the single-GPU row underneath it cuts an eight-step render from four and a half minutes to ninety seconds, and that is the change that reaches anyone actually making things.

VDN-H3 MiniMax H3 Video Delta Attention video-gen open-weights local-creative-ai licensing-provenance

VDN-H3 Renders Video Faster Than It Plays. The Row That Matters Runs on One GPU.

A hybrid-attention rebuild of MiniMax H3 that ships its training code, gets its stability from solving instead of stepping, and starts from a hobbyist's LoRA. Here is what the 74x number actually buys you.

The headline is that VDN-H3 produces a 14.4-second video clip in 11.23 seconds, which is less time than it takes to watch. That is true, and it is also a claim about eight NVIDIA B200s working together, with model loading, warm-up, VAE decoding and MP4 encoding all excluded from the clock. Most people reading about it do not have eight B200s. They have one card, or a rented hour, or a queue on somebody's platform.

So skip the headline and read the row below it. On a single H200, an eight-step render of that same 768p, 14.4-second workload goes from 4.4 minutes on dense MiniMax H3 to 90.5 seconds on VDN-H3. On a single B200, 2.23 minutes becomes 51 seconds. That is the number that changes a working day, because it moves video generation across the line from "start it and go do something else" to "watch it finish."

The team published both rows, in the same table, without burying either. That is worth noticing on its own.

Who built this, and why that matters for the license

VDN-H3 came out of a group split between UC Berkeley, a company called Impossible Inc., and UT Austin, with Kurt Keutzer among the authors. The repo's own footer calls it an "independent architecture study." It is not MiniMax's work, and it is not NVIDIA's, which is worth stating plainly because at least one popular ComfyUI bundle for these weights credits the source implementation to NVlabs/OpenVDN. That repository does not exist. A cache-busted shields.io read on it returns "repo not found." The real code is at OpenVDN/vdn-minimax-h3.

That matters beyond pedantry, because the licensing here splits down the middle and you need to know which half you are reading. The code is Apache 2.0. The weights are not, and the repo says so in as many words: "The model weights are not in this repository and are not covered by that license." VDN-H3 is a derivative of MiniMax H3, so the weights carry the MiniMax H3 Community License Agreement, whose applicable territory excludes the European Union, the United Kingdom, the Republic of Korea and the United States.

Read that list again. It excludes most of the places where this article will be read. I keep flagging this on MiniMax H3 derivatives because the badge you see on the repo you cloned describes the code, and the code is a rounding error against the 82 GB of weights you are about to download.

The mechanism: solving instead of stepping

Attention is the expensive part of video generation, and on a model like H3 the authors put softmax attention at more than 85% of total runtime. The obvious fix is linear attention, which scales far better with sequence length. The obvious problem with the obvious fix is that linear attention loses track of things: subject identity drifts, scene layout wanders, and long-range temporal consistency falls apart.

VDN's answer is to run both and split the work by distance. A sliding-window softmax branch handles nearby frames exactly, in chunks of five consecutive latent frames, where each chunk attends to itself, the chunk before, and the chunk after. A linear branch handles everything further away, running forward and backward so a frame can read a summary of what came before it and what comes after it without double-counting anything in the middle.

Sitting on top of that is the detail I like most, because it is cheap and obviously right in hindsight. Every frame also attends to all tokens in the first and last frames, and both of those boundary frames attend to every other frame. The authors call these 4-way boundary anchors, and they report the cost at 3.57% extra attention density. For that price you get a fixed reference at each end of the clip that the whole sequence can check itself against.

The genuinely new idea is in the linear branch, and it comes down to one word: solve.

Standard linear attention updates its memory one token at a time with the delta rule, which is exactly a gradient descent step on the prediction error. Video models batch that across the spatial tokens in a frame, but batching is not the same as coordinating. In the previous approach the authors compare against, every token in a frame reads from the same frozen memory state and writes its correction without seeing what the other tokens are proposing. Two patches with nearly the same key and different targets cannot negotiate. Their corrections simply pile up.

Video Delta Attention writes down the whole frame as one least-squares problem instead: fit a single new state to all the tokens in this frame at once, with a term that penalizes drifting too far from the memory you already had. Differentiate, and you get a closed form that replaces a batch of independent gradient steps with a matrix inverse.

Here is why that is more than an aesthetic preference. Build a weighted key Gram matrix A from the frame's keys. The old approach passes your inherited memory through I - A, and when many keys in a frame point the same way, the eigenvalues of A grow with the number of tokens. Push one past 2 and I - A starts amplifying instead of contracting, which over a long clip means a stored direction gets louder every frame until the scene corrupts. The usual patch is to scale every key by one over the square root of the frame's token count, which caps the damage.

The joint solve passes memory through (I + A)^-1 instead. Because A is positive semidefinite, every eigenvalue of that inverse lands in the interval from just above zero to one. It cannot amplify, ever, for any number or arrangement of tokens. No fudge factor required.

And the fudge factor was costing something real. A frame-size scale is a blunt instrument: it treats a frame where a hundred tokens all describe the same thing exactly like a frame where a hundred tokens describe a hundred different things. Under the joint solve, repeated evidence strengthens the write (the weight grows with how many tokens agree, while never exceeding one) and independent directions each keep their full write strength instead of being divided by the number of unrelated neighbors sitting next to them. The correction adapts to what the frame actually contains.

The training detail nobody will quote

Dropping a randomly initialized branch into a well-calibrated pretrained model is a good way to wreck it. VDN handles that with a staged pipeline, and one choice inside it is a small masterclass in anticipating your own optimizer.

Stage A1 calibrates each new linear branch one block at a time, 200 steps. Stage A2 installs them all and trains them end to end, 500 steps. Stage B adds LoRA adapters on the QKV and O projections and co-adapts them with the linear branch, 2000 steps. Throughout A1 and A2, the softmax branch, the QKVO projections and the FFN stay frozen.

So does the softmax gate, and the reason given is the good part. Each branch's output passes through a content-dependent sigmoid gate before joining the residual stream. If that gate were trainable early on, the optimizer would find an easy way to reduce loss: turn the softmax branch down rather than learn the linear branch properly. Loss goes down, quality goes with it, and you would not see the problem until much later. Freezing the gate until Stage B removes the cheat.

Then there is a fourth stage the blog post lists separately. The 8-step distillation is trained with DMD2, and it does not start from scratch. It initializes from larryvrh/MiniMax-H3-Turbo-Lora, a community LoRA released August 5 under Apache 2.0 with 926 likes and over 676,000 downloads. A research group at Berkeley built its few-step path on top of a hobbyist's checkpoint and said so in the citation list. That is a healthier picture of how this field works than most release posts admit to.

They also chose eight steps over four on purpose, saying they prioritized quality over latency. Given how many four-step distillations have shipped in the last month, someone deliberately leaving speed on the table is a data point.

Put this into practice

There are three ways in, and only one of them is free.

1. Try the hosted Space first, in about ninety seconds. There is a running demo at hugging-apps/vdn-minimax-h3-demo on ZeroGPU, created September 3, wired to the 8-step checkpoint with a synchronized soundtrack. It costs nothing and it answers the only question worth answering before an 82 GB download: do you like the output. What it cannot tell you is anything about speed, because shared ZeroGPU hardware is nothing like the H200 and B200 the tables describe.

2. Run it locally, if you have the card and the patience. The setup is more particular than most:

conda create -n vdn python=3.12 -y && conda activate vdn
pip install uv
uv pip install torch==2.13.0 --index-url https://download.pytorch.org/whl/cu129
uv pip install --prerelease=allow -e .
bash scripts/setup_diffusers.sh

PyTorch 2.13 and FlashAttention 4 are stated requirements, not suggestions, because the code needs FlexAttention's Flash backend. The --prerelease=allow flag is there for a pre-release CUTLASS dependency and you will need it. The Diffusers install is patched, which is what setup_diffusers.sh handles.

Then pull the weights, and read the layout before you panic at the number:

hf download OpenVDN/vdn-minimax-h3 --local-dir ckpts

That is about 82 GB, but 72 GB of it is the released MiniMax H3 base you may already have on disk. The VDN parts are stage-b-step-2000 at 4.3 GB for the 50-step model and stage-dmd-step-250 at 5.1 GB for the 8-step one.

Your first render is one line, and the two docs disagree about which line. The Hugging Face model card says bash scripts/8nfe_tuned_fp8.sh. The GitHub README says bash scripts/inference/8nfe_tuned_fp8.sh, three separate times. The root-level path returns nothing from the repo; the scripts/inference/ one is real. Use the GitHub form, and note that the multi-GPU variants live in the same directory (8nfe_tuned_fp8_ulysses_h200.sh and _b200.sh). Either way, the first run compiles kernels for several minutes before anything happens, and later runs reuse the cache.

3. Understand the prompt path, because it is not a text box. You cannot hand this a sentence. Prompts get encoded through Qwen3-VL-32B first, into a .pt file, and only then rendered:

python src/inference/encode_prompt.py --prompt "..." --out prompts/mine.pt
python src/inference/infer.py \
  --config configs/inference/8nfe_tuned_fp8.yaml \
  checkpoint=ckpts/stage-dmd-step-250 \
  render.prompt_file=prompts/mine.pt \
  render.out=results/mine.mp4

The authors recommend rewriting your prompt through MiniMax's H3-Context-IR endpoint or its published prompt-writing skills before encoding, and say it improves quality a lot. Take that seriously. Two encoding passes before a pixel exists is a strange workflow, and it is also the workflow.

If you want the ComfyUI route instead, a community bundle exists that lands everything in ComfyUI's own folder names. It gates access behind a form asking your country and asking you to confirm you are not requesting from an excluded territory, which is more honesty about the license than most ports bother with, and it is also a hint about how seriously to take the territory clause.

Where it falls down

The headline is a datacenter number and reads like a product claim. "Generates video faster than it plays" is true at eight B200s and false everywhere else. Two GPUs will not get you there. Neither will four.

The clip length is written four different ways across two documents. The results tables say 14.4 seconds. The blog post's summary paragraph says a 14.3-second video. Another paragraph describes the single-B200 comparison as a 15-second clip. The hero line rounds 11.23 seconds to 11.3. None of this changes the conclusion, and all of it means you should quote the tables rather than the prose.

The speedup does not come free of the training bill. The published gains assume the FP8 optimized path with dedicated kernels and caching. The unoptimized hybrid gets a single H3 block from 332.5 ms to 192.1 ms on a B200; the optimized version reaches 125.3 ms. Roughly half the win is engineering, not architecture, and engineering is the part that will break on your driver.

The environment is fragile by construction. A pre-release FlashAttention, a specific PyTorch build, a patched Diffusers, and a first-run kernel compile measured in minutes. This is research code that shipped honestly, not a package.

The territory clause is not a footnote. Everything above is downstream of weights licensed for a territory that excludes the US, the UK, the EU and South Korea. The code being Apache 2.0 does not help you here.

And the quality claim is qualitative. "Near-lossless" and "nearly indistinguishable" are the authors' words, supported by side-by-side clips they selected. There is no quality benchmark table next to the latency tables. That is not dishonest, and it is not evidence either.

What I would actually watch

The interesting thing about VDN-H3 is not that it is fast. Fast video models arrive weekly now. It is that the entire training stack shipped alongside the weights: all four stages, the configs, the data layout, the preprocessing contract. Someone with a different backbone and a rented cluster can now run this architecture study again on something else. The paper-shaped part of this release is reproducible in a way that almost none of them are.

Which raises the question I cannot answer yet. If the joint-solve linear branch really is stable by construction for any frame geometry, and if it really costs 3.57% extra density to anchor a clip at both ends, then this is not a MiniMax H3 trick. It should port. Whether anyone bothers to port it to a backbone whose license does not exclude four of the largest media markets on earth is a different question, and it is the one I would watch for over the next month.

If you run the single-GPU path on an H200 or a B200 and get numbers that differ from the published table, I want to hear about it. That row is the whole story and almost nobody is going to check it.

Sources

Medium metadata

Title: VDN-H3 Renders Video Faster Than It Plays. The Row That Matters Runs on One GPU.

Subtitle: A hybrid-attention rebuild of MiniMax H3 that ships its training code, gets its stability from solving instead of stepping, and starts from a hobbyist's LoRA.

Tags: AI Video Generation, Machine Learning, Open Source, Diffusion Models, Creative Technology

Suggested publication: Towards Data Science, Better Programming, or self-published

Reading time: about 9 minutes