ffmpeg-skill: A Free AI Video Editor for Your Coding Agent, and Why the Rules It Refuses to Break Matter More Than the 42 Tools

Most of the hours in a video job are not creative. They go to the export that came out at the wrong loudness, the vertical version with the caption sitting on someone's chin, the TikTok file that got rejected for a reason the upload screen will not explain, and the folder of proxies (low-resolution stand-ins for smooth editing) nobody wanted to build. FFmpeg, the free command-line engine underneath a large share of the world's video software, can do every one of those jobs. Almost nobody who edits for a living wants to type its commands.

AI assistants were supposed to close that gap. Ask a chat model for an FFmpeg command and you get one, fast, with total confidence. Sometimes it works. Sometimes it re-encodes your master (rebuilds the video file, losing a little quality each time) three times without saying so, overwrites the original, and tells you everything went fine.

A project called ffmpeg-skill, on this week's Trendshift list of fast-rising GitHub repos, takes a different route. It gives a coding agent (Claude Code, Cursor, Codex, or anything that reads a skill file) a local video editor made of 42 small tools. The tools are useful. The more interesting part is a written list of things the agent is forbidden to do, and that list is the reason I would let it near a client folder.

What it is, in plain terms

ffmpeg-skill is a free toolkit under the MIT licence (use it for anything, including paid work) by the GitHub user kajisho5, sitting at about 2,700 stars. It installs as a "skill," which is a folder of instructions and scripts an AI coding assistant reads before it acts. Its tagline is "Give your coding agent a video editor."

Everything runs on your own machine. There is no cloud service, no account and no API key (the password-like code paid online services require). The scripts use FFmpeg and ffprobe (FFmpeg's file-inspection tool), and nothing else beyond standard Python.

The 42 tools cover the work that eats an afternoon:

• Looking: read a file's real facts (frame rate, colour format, audio channels, HDR or not), find scene changes and beats, and make contact sheets so the agent can see the picture.
• Cutting: trim, join, remove silences, fit to a duration or aspect ratio, crop, stabilise, insert B-roll.
• Sound: clean up voice, duck music under speech, sync two recordings, and normalise loudness.
• Picture: captions (including karaoke-style word highlights), overlays, LUTs (saved colour looks), colour correction, and converting HDR footage for ordinary screens.
• Delivery: platform presets, proxies, spec checks and HTML reports.
• Bigger jobs: render a whole edit from a project file, batch a folder, handle a multicam edit.

It also runs as an MCP server. MCP is the plug standard that lets apps like Claude Desktop call outside tools, and by default the server shows 12 core tools, with all 42 available if you switch them on.

The refusals are the product

Here is what separates this from asking a chatbot for a command. The skill file the agent reads before every job contains a "never" list, and it reads like it was written by someone who has lost a master file. A few of them, quoted from the project's SKILL.md:

• "Never overwrite the source; write new files next to input or where asked."
• "Never guess a raw ffmpeg/ffprobe invocation outside scripts/*.py."
• "Never ask for what probe.py can tell you."

Other rules, which I am paraphrasing: never invent dialogue for captions, never rewrite caption text to make it fit, never claim to have looked at a picture it did not look at, and never report success without inspecting the output file.

That last one is the habit chat assistants lack. The skill treats a step as done only when the script finished cleanly and the new file checks out against what you asked for: duration, resolution, frame rate and audio. After any change to the picture, the agent has to make a contact sheet, a grid of still frames, and it is not allowed to finish until it names that image. If it spots a judgement problem, like a face cut off by a crop or text sitting over someone's mouth, it has to tell you in one line rather than silently pass or fail.

Each tool is also a script with fixed, typed options. The agent cannot hand FFmpeg a free-form string of filters, which is where most AI-written commands go wrong. It picks a named tool, passes it named settings, and reads back the result.

This is the mechanism worth stealing for any AI tool you use: the model is clever, but the guard rails are written down, in plain language, where you can read them.

Delivery is where it earns its keep

The single most useful tool for a working editor is probably the spec checker. ffmpeg-skill ships export presets for YouTube (including 4K, HDR and AV1 versions), Reels, TikTok, Shorts, LinkedIn, Facebook, X, plus ProRes and H.265 masters and GIFs. A checker tests a finished file against the destination's requirements and gives a fix for each failure.

The loudness targets are the ones engineers actually use: −14 LUFS for YouTube and social, −16 for podcasts, −23 for broadcast. (LUFS is the standard unit for perceived loudness; it is why one video sounds twice as loud as the next in a feed.) If you have ever re-exported a piece because a client's phone made it sound quiet beside everything else, this is the fix.

It also prefers not to re-encode. When nothing about the picture has to change, it copies the video stream as-is, which is fast and loses no quality. It only re-encodes when the job needs it: a cut that has to land on an exact frame, a filter, or a format change. That one rule protects your quality over a chain of edits better than any setting you will remember to tick.

Put this into practice

Start with a job you would happily hand to an assistant: one finished edit that needs four platform versions.

1. Install FFmpeg first. Version 6.1 or newer is recommended (5.0 still works but is marked as deprecated). On a Mac, the project says the plain Homebrew ffmpeg lacks some filters, so install ffmpeg-full.
2. Install the skill. For Claude Code, run npx ffmpeg-skill. Add --cursor or --codex for those tools, or claude plugin install kajisho5/ffmpeg-skill to install it as a plugin. Then run npx ffmpeg-skill doctor once to see what your FFmpeg build is missing.
3. Work on copies in a fresh folder. The skill never overwrites sources, but the habit costs nothing.
4. Ask for the destination, not the settings. "Make YouTube, Reels and LinkedIn versions of master.mov, normalise loudness for each, and check them against spec." Let the agent probe the file instead of describing it yourself. The skill tells it not to ask for anything the inspection can answer.
5. Read the report, then look. Open the contact sheets it made. If it flagged a cut-off face on the vertical crop, that is your call to make, not its.
6. Keep the project file. For bigger edits, the skill can render from a JSON project file. That file is a written record of every cut and setting, which means you can change one line and re-render instead of re-describing the whole edit.

Once that works, move to the chores you hate most. Silence removal on a talking-head recording, captions from a real transcript, proxies for a folder of camera files overnight.

Where it breaks

The project is unusually frank about its limits, and you should take them at face value.

It cannot see who is in the frame. There is no face or subject detection. When it reframes a horizontal video to vertical, it centre-crops unless the agent picks an anchor point from the contact sheet. On a two-person interview, that can mean cutting both people in half. Check every vertical.

It does not remove things properly. The logo-removal mode is a blend from the surrounding border, not AI fill that repaints the area, and it cannot find the logo for you. Moving logos and burned-in subtitles are out of scope.

Its taste is average. It will remove silences by rule and cut to a beat by detection. It will not know which pause was the moment. That makes it a delivery and prep tool, not an editor with judgement.

Hardware speed-ups are narrow. Graphics-card-accelerated exporting is optional and only covers Apple Silicon Macs. On a Windows or Linux machine, expect ordinary export times.

Windows is less proven. The README notes that the full real-device test set has not run on Windows and that an install by someone other than the maintainer has not been reproduced there yet. Mac and Linux users are on firmer ground.

The agent is not free. The skill costs nothing, but the coding assistant driving it runs on your subscription or API bill. The project's own testing, the "eval 22" round on version 1.18.3 in September, routed 7 of 8 new prompts to the right tool, with the eighth an honest partial result. Good, not flawless, and that was measured by the maintainer.

It is a young project moving fast. Version 2.16 is current, with breaking changes promised only at 3.0, when FFmpeg 6.1 becomes the minimum. Expect things to shift.

The bigger point

A lot of the AI video conversation is about generating footage. The less glamorous truth is that most editors would get more hours back from something that handles the last mile reliably: the right loudness, the right file format, the right crop, the file that does not get bounced.

What ffmpeg-skill shows is how to make an assistant trustworthy at that work. Not by making it smarter, but by writing down what it may never do and making it prove each step before it says "done." You can read those rules in ten minutes, which is more than you can say for most AI features buried inside paid editing apps.

Try it on one deliverable you already know cold. Give the agent your master and a list of platforms, then compare its four exports with the ones you made last month. If the loudness, crops and specs match yours, you have just handed off the most boring hour of your week. If they do not, the report will tell you exactly which rule it followed and why, and that is a better place to start than a confident chatbot that never checked.

---

• Sources: ffmpeg-skill on GitHub · ffmpeg-skill README · SKILL.md · Agent Skills docs