Analysis September 1, 2026 tool 10 min read

Matrix-Game 3.5 is a genuinely useful open camera-controllable world model whose Apache 2.0 badge does not survive contact with its own quickstart, because the depth checkpoint both entrypoints require is CC BY-NC 4.0.

Matrix-Game 3.5 Depth Anything 3 Wan2.2-TI2V-5B Riemann Dynamics video-gen open-weights licensing-provenance local-creative-ai creative-workflows

Matrix-Game 3.5 Is Apache 2.0. The Depth Model It Cannot Run Without Is Not.

An open world model that lets you walk a camera through a generated scene, and the three things standing between you and a commercial render that are not printed on the label.

The licence line at the bottom of the Matrix-Game 3.5 repository is short and clean: "This project is licensed under the Apache License, Version 2.0 ..." Scroll up eleven lines to the Model Download section and it tells you to fetch three checkpoints before anything will run. Two of them are permissive. The third is depth-anything/DA3NESTED-GIANT-LARGE-1.1, and its Hugging Face card carries license: cc-by-nc-4.0.

Non-commercial. Not a footnote, not a rider, not a clause buried in a PDF. The plain licence tag on a model that both of Matrix-Game 3.5's entrypoints require in order to produce a single frame.

I want to be careful about what that does and does not mean, because licence stacking is the place where open-weights coverage most reliably goes wrong in both directions. Riemann Dynamics has not done anything sneaky. Their code is Apache 2.0 and their weights are Apache 2.0, and they name the dependency openly in the README. But the thing a creator actually cares about is not the licence on any single file. It is whether the pipeline they just spent an afternoon assembling can be pointed at paid work. For Matrix-Game 3.5 as documented, the honest answer is no, and you will not learn that from the badge.

What the model actually does

Set the licensing aside for a moment, because the capability is real and it is not one you had last month.

Matrix-Game 3.5 takes three inputs: an anchor image, a camera trajectory, and a text prompt. It gives you back 720p video that moves through the scene along that trajectory, in first person or third person. What separates it from a normal image-to-video model is that it remembers geometry. Pan away from a bookshelf and pan back, and the bookshelf is still the bookshelf.

The mechanism has two named parts. Patch Memory lifts already-generated observations into a 3D memory and retrieves from it based on what should be visible from wherever the camera has moved to. Warped PRoPE folds the camera projection matrices into the model's positional encoding, so the network reasons about time and viewing angle in the same operation instead of treating the camera as an afterthought. Riemann Dynamics describes both as "parameter-free," meaning the memory is machinery bolted around the backbone rather than more weights inside it.

Alongside that runs a split the team calls a static-dynamic decoupled world representation. Static scenery is held by the geometric memory. Moving subjects are held by separate lightweight reference tokens that carry identity and appearance. That split is why third-person mode accepts up to four protagonist crops and locks the character to them. Drop the --refs argument and the model invents a protagonist freely, which is fine for scouting and useless for anything with a cast.

Both base models are 5B parameters and built on top of Wan2.2-TI2V-5B, which supplies the T5 text encoder, the VAE, the DiT scaffold and the umt5-xxl tokenizer. Wan2.2-TI2V-5B is Apache 2.0, so that leg of the stack is clean.

The three-checkpoint download, and where the licence breaks

Here is the actual checkpoints/ layout the README asks you to build:

checkpoints/
├── Wan2.2-TI2V-5B/              DiT shards + T5 encoder + VAE + tokenizer
├── DA3NESTED-GIANT-LARGE-1.1/   depth estimator
├── first-person.safetensors     Matrix-Game-3.5 first-person model
├── third-person.safetensors     Matrix-Game-3.5 third-person model
└── distilled-first-person.safetensors  distilled three-step first-person model

Three licences in one folder. Apache 2.0 on the Matrix-Game weights. Apache 2.0 on Wan2.2-TI2V-5B. CC BY-NC 4.0 on Depth Anything 3.

The depth model is not optional and it is not swappable in any documented way. The README lists Depth Anything 3, alongside Wan2.2-TI2V-5B, as "shared dependencies for both inference paths," and describes its job as "metric depth for Mosaic Memory." Metric depth is the thing that makes Patch Memory work at all: without knowing how far away a surface is in real units, you cannot reproject an old observation into a new camera view. Take the depth model out and you do not get a degraded version of Matrix-Game 3.5. You get a model with no memory, which is the entire feature.

Both entrypoints accept a --da3-dir flag and a DA3_MODEL_PATH environment variable, so the code will happily load a different checkpoint from a different folder. That is an escape hatch for someone who has a commercially licensed metric depth estimator with a compatible interface and the appetite to validate it. It is not a path most people making things will walk.

There is a second, softer note in the same area. The bundled first-person sample scenes come from SANA-WM-Bench under CC BY 4.0, which is attribution-required rather than non-commercial. Not a blocker, but if a sample scene ends up in a portfolio piece, it needs a credit.

The distilled model, and what "three steps" actually costs

The phrase attached to Matrix-Game 3.5 everywhere is minute-long real-time generation. That belongs to one specific artifact: RiemannDynamics/Matrix-Game-3.5-Distilled, a three-step first-person causal checkpoint. The base models you download for third-person work are bidirectional and run 25 denoising steps at guidance scale 5.0 by default. Two different animals, one headline.

The distilled model has a wrinkle that deserves more attention than it has had. From DISTILLED_INFERENCE.md:

The released student was distilled with CFG scale 3 and requires the same CFG transition at inference: every denoising step evaluates conditional and unconditional student branches and combines them with the configured scale.

Read that carefully. Classifier-free guidance is usually the thing distillation removes, because evaluating two branches per step doubles the work. This student was trained with guidance baked into its schedule, so it needs guidance at inference too. Three steps therefore costs roughly double what three unguided steps would, since every step runs a conditional and an unconditional branch. Whether those land as two calls or one batched call of two is an implementation detail the docs do not state, but the compute is the same either way. Running it conditional-only "changes the learned transition and reduces quality," so this is not a knob you can turn off to buy back speed.

That does not make the distilled model bad. It is still roughly a sixth of the base model's step budget. It does mean that if you were sizing a rental GPU on the assumption that three steps behaves like three ordinary steps, your estimate is about half of what you need.

The distilled path also carries a small operational tax the base path does not: infer_distilled.py requires an explicit --checkpoint, an explicit --config, and exactly one text-conditioning source out of --prompt, --prompt-file or --caption. Unknown configuration keys fail immediately rather than being ignored. That last behaviour is good engineering and will still bite you the first time you copy a YAML file from the wrong example.

The camera file is the real barrier

Everyone writing about this model leads with the 40 GB VRAM requirement. The README is blunt about it: one NVIDIA GPU with at least 40 GB of VRAM, Linux, at least 64 GB of system RAM, Python 3.10, and 704x1280 generation peaking around 40 GB. That is a rented A100 or H100 rather than anything on your desk, and it is a solvable problem. You can pay for it by the hour.

The barrier nobody mentions is the input format. --camera takes a .npz file containing extrinsics_c2w, an (N,4,4) array of camera-to-world matrices with metric translation, plus intrinsics as (N,4) giving [fx,fy,cx,cy] in pixels of the anchor image.

There is no text field for the camera. You cannot write "slow push in, then arc left." You need a numerical trajectory, one 4x4 matrix per pose, in a coordinate convention that matches your anchor image's optics. And you need a lot of them: each generated block is 80 frames and consumes 84 camera poses, so a trajectory must be at least 1 + 84 × num_blocks long. Come up short and the model pads by holding the final pose, which reads on screen as a camera that stops dead partway through your shot.

The repository ships sample .npz files and nothing to author new ones. In practice you are exporting a camera path out of Blender or another 3D application, converting the matrices into the expected convention, and writing your own .npz. Riemann Dynamics did include --camera-convention w2c for people whose extrinsics are world-to-camera, which is a small kindness that tells you they know this is where people will get stuck.

This is worth naming plainly because it changes who the tool is for. Matrix-Game 3.5 is not a prompt-and-see tool that happens to need a big GPU. It is a tool for people who already work in 3D and want a renderer that invents the world instead of shading one they built. If that describes you, this is a genuinely interesting piece of software. If it does not, the VRAM number is the least of your problems.

Put this into practice

If you want to try it, here is the shortest honest path.

Decide the licence question first. If the output is going anywhere commercial, stop and either source a commercially licensed metric depth model to point --da3-dir at, or treat this as a research and personal-work tool. Do this before you rent a GPU, not after.

Rent, do not buy. Any 40 GB or larger NVIDIA card on Linux. Set up Python 3.10, install PyTorch matching your CUDA version, then pip install -r requirements.txt. The repo vendors its third-party sources, including the DiffSynth-based pipeline, the frustum reprojection engine and Depth Anything 3, so there is no flash-attention compile step. That saves a real amount of pain.

Run the bundled sample before your own data. The first-person case is one line:

python infer.py --person first \
    --image  samples/first_person/case_7/input.png \
    --camera samples/first_person/case_7/camera.npz \
    --prompt-file samples/first_person/case_7/prompt.txt

Then look at the second output file, not the first. Every run writes result.mp4 and also memory_visualization.mp4, a two-row diagnostic panel showing the generation against the mosaic memory. Third-person runs add subject_ref_preview.jpg, the protagonist reference canvas. That memory visualization is the most useful thing in the repository and almost nobody will open it. When a generation drifts, it shows you whether the memory retrieved the wrong geometry or the generator ignored the right geometry. Those failures have different fixes, and without the panel you are guessing.

Author the camera last. Get a sample working end to end first. Then export a path from your 3D application, check your convention, and remember the 1 + 84 × num_blocks pose floor.

For third person, supply reference crops. --refs takes a directory of protagonist crops with optional *_mask.png files; without masks, full-white masks are assumed. Identity is locked when you supply them and invented when you do not.

Where it breaks

The 40 GB floor is a floor, not a target, and it is quoted for 704x1280. Linux only. 64 GB of system RAM.

The base model generates one 80-frame block by default. Longer output means more blocks and more camera poses, and the technical report's long-horizon claims are made about the distilled causal path rather than the 25-step bidirectional one you will most likely start with.

There is no hosted demo. No Space, no playground, nothing to click. For a model with weights this accessible, that is an odd gap, and it means your first look at the output quality costs you a GPU rental.

The release history is worth knowing about too. The weights have been on Hugging Face since July 18 for the base models and July 26 for the distilled one. The code, the technical report and the project site came at the end of August, with the paper dated August 30. For roughly six weeks there were downloadable checkpoints and no way to run them. If you saw this model mentioned in July and moved on, it is worth a second look now, because the thing that changed is the part that matters.

And the licence again, because it is the one that will cost somebody real money: the badge on the repository describes the repository. It does not describe the pipeline.

What I would actually do

If you already work in 3D and have a camera path pipeline, rent an H100 for an afternoon, run the sample, then run one of your own trajectories, and spend most of your time watching memory_visualization.mp4 rather than the result. You will learn more about where generative world models are in September 2026 from that diagnostic panel than from any demo reel.

If you do not work in 3D, wait. Not because the model is weak, but because the missing piece is a camera authoring tool, and somebody is going to build one. When they do, this model gets dramatically more interesting to dramatically more people.

Either way, open the licence tags on every checkpoint a project tells you to download. This one is Apache 2.0 in the place people look and CC BY-NC 4.0 in the place they do not, and that pattern is going to keep showing up as open models get assembled out of other people's parts.

Medium metadata

Title: Matrix-Game 3.5 Is Apache 2.0. The Depth Model It Cannot Run Without Is Not.
Subtitle: An open camera-controllable world model, and the three things standing between you and a commercial render that are not printed on the label.
Tags: AI Video Generation, Open Source AI, World Models, Software Licensing, Generative AI
Estimated read: 8 minutes
Canonical sources: Matrix-Game 3.5 repository · Base weights · Distilled weights · Depth Anything 3 checkpoint · Wan2.2-TI2V-5B · Project site