How to Reverse-Engineer Any Video Shot by Shot With rea-video, a Free Tool That Turns a Reference Into a Shot List
Every editor keeps a folder of videos they love and cannot quite explain. The ad that feels fast without feeling frantic. The talking-head clip that somehow holds attention for ninety seconds. When a client sends one over with "make it feel like this," you watch it six times and still end up guessing.
Here is the uncomfortable truth about "feel": most of it is measurable. How long each shot lasts. Where the cuts land against the music. When text pops on and how long it stays. Whether the camera drifts or sits still. Those choices add up to the mood, and you can count them.
A new open-source project called rea-video does the counting for you. Its README opens bluntly: "Reverse-engineer any video." Give it a video file and it hands back a shot-by-shot report, the separated voice and music, the on-screen captions as subtitle files, and a plan for remaking each shot. It runs on your own computer, and it costs nothing unless you choose to plug in a paid AI model.
I think this is one of the most useful things AI can do for an editor right now. Not making the video. Explaining one.
What you get back
Point rea-video at a file and it builds a folder of results. The ones that matter most:
- report.html, a page you open in your browser with every shot laid out, including its first and last frame. This is the part you will actually read.
- breakdown.json, the same information in a data file other tools can use.
- storyboard.jpg, one image showing the whole video as a strip of shots, which is perfect for pinning next to your edit.
- Caption files (SRT and ASS, the two common subtitle formats) rebuilt from the text that appears on screen.
- Separated audio: the original track, the vocals and the music as separate files, so you can hear exactly where the beat sits under the voice.
The README describes the remake plan as being "split into what to film, what to generate and what to code." In plain terms, it sorts every shot into something you would shoot with a camera, something you would make with an AI video tool, or something you would build as motion graphics.
The project was created on October 29 and had 161 stars on GitHub when we checked on November 5. It is young. Treat it accordingly.
How it works, in plain terms
rea-video stands on two familiar pieces of software: ffmpeg, the free command-line tool that sits under half the video apps you already use, and Python.
First, it cuts the video at every shot change and saves a frame from the start and end of each shot. It reads the text that appears on screen, frame by frame, so it can tell you when a caption appears and disappears. It tracks camera movement and the beat of the soundtrack. It then splits the audio into voice and music using a small separation model, about 67 MB, which downloads the first time you run it.
All of that happens offline. Optionally, you can connect an AI model that can look at images, such as Google's Gemini, to describe each shot in words: the lighting, the camera, and a prompt you might use to recreate it. The README says it matters most on crunchy files.
Here is the tool's own measure. It labels every shot by how it was made: filmed, AI-generated, 3D or code. On ten videos re-encoded by WeChat (the kind of recompressed file you get when someone forwards a clip), the README reports 88.6 percent of shot time labeled correctly with the built-in tools alone and 99.6 percent with Gemini added. On original files, the tools alone score far higher. Those are the project's own tests on a small set, not an independent review.
Put this into practice
You will need ffmpeg and Python 3.10 or newer, which is all the README lists. The commands below come from the project's command-line guide.
1. Install ffmpeg. On a Mac with Homebrew:
brew install ffmpeg
On Ubuntu, apt install ffmpeg.
2. Get the tool. Download the repository, open a terminal in its folder, and run:
pip install -e ".[ocr]"
If you work in Claude Code, you can also add it as a plugin with /plugin marketplace add JuneYaooo/rea-video. You still need the install step above.
3. Break down your reference.
rea-video decompose ref.mp4 -o out/ref --lang en
The --lang en part asks for English reports; the project is Chinese-first, so set it every time.
4. Open the report.
open out/ref/report.html
That command is for Mac; on Windows or Linux, open the file from your file browser.
5. Split the sound if you want to study the music edit separately:
rea-video stems ref.mp4 -o out/ref
6. Optional: add an AI model that can see. If you have a key for Gemini or another AI service that accepts OpenAI's standard connection format, set three variables and add --vlm (short for vision-language model, an AI that reads pictures as well as text):
export REA_VLM_BASE_URL=https://.../v1 REA_VLM_API_KEY=... REA_VLM_MODEL=gemini-2.5-pro
rea-video decompose ref.mp4 -o out/ref --vlm --lang en
This step costs whatever your AI provider charges. Everything before it is free.
How to read the report like an editor
The report is only useful if you turn it into decisions. Here is the routine I would use on any client reference.
Count the shots, then divide. Total running time divided by the number of shots gives you the average shot length. That single number explains more about "pace" than any adjective in a brief. A thirty-second spot with twenty shots is a different animal from one with eight.
Look for the rhythm, not the average. Scroll the storyboard and notice where shots get shorter. Most good edits speed up into a moment and breathe after it. Mark those spots.
Line up the cuts with the music file. Play the separated music track while you scroll the report. You will hear whether the editor cut on the beat, just before it, or deliberately against it.
Study the captions as a design system. The caption files tell you how often text appears, how long it stays, and how many words it holds. Copy the system (say, four words at a time, held for a second) rather than the words themselves.
Write your own shot list from the breakdown. Keep the structure: number of shots, their lengths, where text appears, where the music hits. Replace every image and every line with your own. That is the deliverable, a skeleton you fill with original material.
Where it breaks
This is a promising tool from one author, a week old, and it has real gaps.
Check the license yourself. The repository carries an Apache 2.0 license file, which allows commercial use with attribution, but GitHub's own metadata did not detect it when we checked and the README never mentions it. Read the file before you build the tool into anything you sell.
Most of the documentation is in Chinese. The main README is in English, but the detailed docs, including the command guide, are in Chinese "for now," per the README. The commands themselves are readable, and a translation tool covers the rest, but expect some friction.
The remake side is unfinished. The "code" part of the plan gives you text tracks and layout, but by the project's own status notes it does not yet build a motion graphics project for you. The automatic remake command, rea-video remake-plan, feeds a separate repository by the same author and calls paid AI video services. For most creators, the report is the product and the remake step is a curiosity.
Labels slip on rough files without help. The 88.6 percent figure on recompressed clips means roughly one shot in ten may be mislabeled (filmed versus AI versus code) when you skip the AI model. Spot-check the shots you care about before you build a shot list on them.
It explains a video. It does not give you permission to copy it. The README says it directly: "Don't reuse other people's footage, music, characters or likeness." Separating a commercial song from someone's ad does not make that song yours. Learn the structure, then make your own material.
The reference was always a puzzle
Editors have always reverse-engineered the work they admire. We used to do it with a stopwatch, a notepad and a lot of scrubbing back and forth. rea-video does the tedious counting so you can spend your time on the part that needs judgment: deciding which of those choices fits your story and which ones only worked for theirs.
Pick one reference you have been meaning to study. Run it through rea-video, divide the running time by the shot count, and line the cuts up against the music. Then write a shot list that keeps the rhythm and throws away everything else. If the result feels like the reference without looking like it, you have learned the only thing worth taking from it.