FervorCreative AI
Live Latest 18.11.26 · morning 401 tools tracked 992 workflows indexed 626 topics Hot: Gaussian splatting, ComfyUI, ZONOS3-Mini

A generated clip with a moving camera already holds enough information to rebuild the scene in 3D, and the free ai-video-in-code recipe turns one good five-second take into new camera moves without generating again.

ai-video-in-codeGaussian splattingBrushCOLMAPSeedancevideo-gencreative-workflowslocal-creative-aiai-editing

Turn a Five-Second AI Video Clip Into New Camera Angles: A Free Workflow With ai-video-in-code

Every AI video clip comes with one camera move you cannot change. A new open-source recipe turns that clip into a 3D scene on a Mac in about fifteen minutes, then films it again from angles the clip never had.

An AI video clip is a strange kind of footage. It looks like a camera moved through a real place, but there was no place and no camera. There is only the clip. So when you need the reverse angle, the slow push-in for a title card or two more seconds of the same shot, the normal answer is to generate again and hope the room, the light and the objects come back the same. They usually do not.

Here is the thing most people miss. If the camera in that clip moves even a little, the clip already contains the shape of the scene. Every frame shows the same objects from a slightly different spot. That is exactly the raw material that 3D scanning has used for years. Scanning software does not care whether the frames came from an iPhone or from a video model. It only cares that the views agree with each other.

That is the idea behind ai-video-in-code, a free recipe published on GitHub on November 17 by Sirio Berati under the MIT license. Give it a five-second clip. It works out where the camera was in every frame, builds a 3D version of the scene, and lets you render new camera moves through it: an arc, a push-in, a rise, a reverse. The author's examples were made from clips generated with ByteDance's Seedance, but any clip with the right qualities works.

My take: this is the most practical thing I have seen this month for anyone cutting AI footage into real edits. It gives you coverage from a take you already love, and it is honest about the shots it cannot handle.

How a flat clip becomes a 3D scene

The pipeline has four stages, and only the original clip involves AI. On the author's Apple Silicon Mac none of it needs an NVIDIA card, and Brush, the trainer, also runs on AMD, Intel and Apple graphics.

First, it pulls frames. The free tool ffmpeg grabs every second frame, so a five-second clip becomes about 60 still pictures.

Second, it finds the camera. This step is called camera tracking, or the "solve." A long-standing free tool called COLMAP compares the stills, spots features that appear in several of them (the corner of a book, a leaf, a label) and works out where the camera must have been standing for each frame. It also produces a rough cloud of 3D points. This takes under two minutes.

Third, it builds the scene out of blobs. This is where Gaussian splatting comes in. Picture the scene rebuilt from hundreds of thousands of tiny soft colored blobs, each with a position, a size, a color and a transparency. A trainer called Brush (open source, Apache 2.0) starts with rough blobs, renders them from one of the camera positions COLMAP found, compares that render with the real frame, and nudges every blob to shrink the difference. It repeats that 5,000 times. Where the picture is still wrong, it adds more blobs. On the author's M3 Pro MacBook, this takes 12 to 19 minutes.

Fourth, it films the scene again. A small web page draws the blobs from any camera path you choose, frame by frame, and ffmpeg stitches them into an MP4. A five-second move renders in about 20 seconds.

At the end you have two things: a new shot and a 3D file of the scene you can keep coming back to. The README puts it plainly: "Only the clip is AI." Everything after it is ordinary math and an ordinary renderer.

Why this beats rerolling

A video generator gives you a performance. You cannot ask it for the same performance from six feet to the left. That is the gap this fills.

Say you generated a gorgeous five-second orbit around a table of plants for an opening sequence. The edit needs a slow push-in under the title and a reversed move for the outro. Rerolling means new plants, new light, a new table. Rebuilding the scene means the same plants, the same light, the same table, from new positions. For a product shot, a set for an explainer or an establishing shot in a short film, consistency is worth more than a fresh roll of the dice.

It is also cheap. If you pay per second of generated video, every reroll costs money. Here, the clip is paid for once. Every extra camera move costs about twenty seconds of your laptop's time.

Put this into practice

The recipe is tested on one setup: macOS 15 on an Apple Silicon Mac. You will use the Terminal, but only to paste commands.

  1. Install the tools. With Homebrew (a free Mac installer for command-line tools) set up, run brew install ffmpeg colmap node, then pip3 install numpy. You also need Google Chrome, which the renderer uses in the background.
  2. Get the recipe. Run git clone https://github.com/sirioberati/ai-video-in-code, move into the folder with cd ai-video-in-code, then npm install.
  3. Get the trainer. Run ./tools/get_brush.sh. It downloads Brush version 0.3.0 (about 50 MB) and checks it is the real file. Use this version; the README warns that others changed their options.
  4. Pick the right clip. This matters more than any setting. You want about five seconds, a camera that clearly moves (an orbit, a slide or a push), and plenty of texture: wood grain, labels, plants, books. Save it as footage/plants.mp4. The file name becomes the scene name.
  5. Do a quick test run first. Start the small local server in one Terminal window with python3 tools/serve.py 5880 . and leave it running. In a second window run ./run_clip.sh plants 1000. The 1000 asks for a short training run, a fraction of the full 5,000 steps, so you find out quickly whether the camera solve worked.
  6. Read the camera line. The script prints a line like cameras 61 points 10325. If the cameras number is close to your frame count, the solve worked. If it is far lower, pick a different clip before you spend the full training time.
  7. Do the full run. ./run_clip.sh plants trains all 5,000 steps and writes export/plants-new-camera.mp4.
  8. Design your own moves. Use render.mjs with a few plain settings: push (how far the camera moves forward), lift (how far it rises), tilt (how much it angles down), and u0 and u1 (where along the original path the move starts and ends; swap them to reverse the move). The README has a copy-and-paste example.
  9. Cut a side-by-side. This is the best quality check you have. The recipe can also re-render the clip from its own original cameras. Put that next to the source clip. If the two look alike, your scene is solid and your new moves will hold up.

Where it breaks

The camera has to move. If the clip is a locked-off shot, COLMAP has nothing to compare and the run stops. A lot of AI video has a nearly still camera, so check before you fall in love with a clip for this.

Plain surfaces turn to smoke. The README shows a white mug on a white wall going foggy, because there is nothing for the camera solve or the trainer to hold onto. Clean studio backgrounds, smooth skin and clear skies are the weak spots.

You cannot go where the clip did not look. New moves only work inside the part of the scene the original camera saw. Push too far, rise too high or start a move at the very edge of the clip and the frame turns black or blurry at the sides. Keep moves small at first, and keep u0 and u1 between about 0.1 and 0.9.

Shine goes flat. The renderer in this repo draws each blob's base color only, so reflections and glossy highlights that shift as the camera moves will not shift. Glass, chrome and wet surfaces look duller than in the source.

Nothing in the scene moves. This rebuilds a frozen moment. People walking, leaves blowing and steam rising will smear or ghost, because the trainer assumes the world held still while the camera moved (standard splatting rebuilds a static scene). Pick clips where only the camera is moving.

It is a fresh, single-platform recipe. It had 5 stars the morning after release and has been tested on one Mac. The setup script also downloads a Linux build of the trainer, but Linux has not been tested. Windows needs extra manual steps the README sketches but has not tested. Rerunning a scene name replaces its folder, so copy anything you want to keep.

The AI part still has rules. The new shots inherit whatever terms came with the source clip. Check your video service's terms before the rebuilt shots go into paid work.

The take is now a location

The way most people use AI video treats every generation as a finished shot: you keep it or you throw it away. This recipe suggests a different habit. A good generated clip with a moving camera is closer to a location scout than a shot. You came back with enough footage to know the place, and now you can stage new angles in it.

Go through your folder of AI clips and find the one with a slow camera move, a lot of texture and nothing moving except the camera. Run the 1,000-step test on it tonight. If the cameras line comes back full, you have a set you can shoot again as many times as the edit needs.


From the briefing