Automate The Mechanical Half Of Video Editing With Claude Code

Editing is where most creators lose their week. Not filming, not writing, editing. And the reason it eats so much time is that a single edit mixes two completely different kinds of work: mechanical labour that any machine could do, and judgment that no machine can do yet. Nobody separates them, so the whole job gets treated as skilled work and priced in hours. Pull the two apart and roughly two thirds of an edit turns out to be mechanical.

Before you build anything, sort every task in your edit into two piles. This sounds like planning theatre. It is actually the decision that determines whether the pipeline you build saves you time or quietly makes your videos worse.

Mechanical work is anything where a correct answer exists and it is written down in the file. Silence is mechanical: the waveform drops below a threshold for longer than a set duration. A blown take you immediately redid is mechanical: the transcript contains the same sentence twice. So are caption timing, burning text into frames, and naming and moving files.

Taste work is anything where two competent editors would disagree. Which of three good takes has the most energy. Whether a pause is dead air or a beat that lands. Whether the hook is any good. Machines are excellent at the first pile and genuinely bad at the second, and no prompting changes that, because the information the decision needs is not in the file.

The shape of this system is a folder, not an app. That choice matters more than it looks, because a folder is the only interface that survives being run on a schedule while you are asleep.

Three subfolders inside one project directory:

• raw, where you drop anything you filmed. This is the only folder you ever touch by hand.
• outputs, where finished captioned files land, named so you can find them later.
• processed, where raw files get moved after a successful run so the next run does not redo them.

That third folder is what turns a script into a system. Without it, every scheduled run reprocesses everything you have ever filmed and you burn an hour of compute to produce files you already had. The move-on-success step is the difference between a pipeline and a loop that eats your laptop.

Open Claude Code in that project directory and switch to plan mode before you describe anything. It writes out the whole build for you to read before a single file is created, which is where you catch the misunderstanding that would otherwise cost you an hour of debugging.

Every downstream decision reads from a transcript, so transcription happens first and everything else derives from it. Use faster-whisper, a local reimplementation of Whisper. It runs on your machine, costs nothing per video, and needs no API key, so a scheduled overnight run cannot fail because a balance ran out.

What you want back is not a block of text. It is a word-level or segment-level transcript with timestamps, because timestamps are what let the next stage translate a sentence you want gone into a range of seconds FFmpeg can remove. A transcript without timing is a document. A transcript with timing is an edit decision list.

Two separate operations get confused constantly. Retake removal reads the transcript and looks for repetition: a sentence started twice, a false start, a flub followed by a restart. Silence trimming reads the audio and removes gaps longer than a threshold. Different inputs, different failure modes.

Do retakes first. Trim silences first and you destroy the pauses that mark where one attempt ended and the next began, which is the retake detector's best signal. Cut the bad attempts out of a file that still has its natural spacing, then tighten what remains.

The silence threshold is the one number worth tuning by hand, and it depends entirely on the platform:

• Short vertical content: cut aggressively. Anything over roughly a quarter second of dead air can go, and the result should feel slightly breathless.
• Long-form talking head: cut gently. Removing every pause makes a ten minute video exhausting to watch and strips out the emphasis pauses that carry meaning.
• Interview or two-person footage: be careful. Silence between speakers is content, and an aggressive trim makes people sound like they are interrupting each other.

The fork here is worth naming explicitly, because it is the one people get wrong: aggressive trimming is right for short form and wrong for everything else. Set it per output format, not once globally, and if you only ever make reels, set it aggressive and forget it.

The text on screen for the first few seconds is not a subtitle for your opening line. It is a second, parallel hook, and it works because the viewer receives two different pieces of information at once.

The rule is simple and almost everyone breaks it. The overlay and the spoken hook run simultaneously, so they must each stand alone and say something different. An overlay that transcribes what you are saying spends your most valuable screen real estate on redundancy. The goal is intrigue, not description. A good overlay makes the viewer ask a question rather than answering one.

Three patterns that reliably do that:

• The paradox. Something that sounds wrong or self-contradictory, so the viewer stays to resolve it. "She was right." "I fired all three." "It cost me everything."
• The social proof gap. An implication that other people already knew this and you did not. "78,000 people knew this before me."
• The confession. An admission that creates tension about what follows. "I almost didn't post this."

Bake these patterns into the pipeline as a written spec rather than choosing an overlay yourself every time, and set the physical constraints alongside them: big bold type across roughly the top third of the frame, plain white on a solid black bar with no transparency, visible for the first six seconds. Solid beats semi-transparent, because a transparent bar over moving footage becomes unreadable the moment you move.

This is the single most common structural mistake in a build like this, and it separates a pipeline that works from one that produces captions drifting further out of sync with every passing second.

You transcribed the raw file in stage two. Then you cut retakes and trimmed silences, which removed time from the middle of the file. Every timestamp in that transcript now points at the wrong moment, and the error compounds: by the end of a two minute clip the captions can be several seconds adrift. A fixed offset does not fix it, because the removals were scattered throughout.

The fix is to run transcription a second time, on the edited file, and derive caption timing from that. It costs a second pass of compute and it is not optional. Caption timing must come from the file you are actually shipping, not from the file you started with.

While you are there, give the pipeline permission to correct the transcript rather than reproduce it faithfully. Speech recognition mangles brand names and technical terms constantly, so a video about Claude ends up captioned as claw. Nobody is checking your captions against a court transcript. They are checking whether your product name is spelled right.

Default auto-captions look like auto-captions, and viewers have learned to read that look as low effort. The style that outperforms is smaller and quieter than most people expect. The settings worth specifying in the pipeline:

• Smaller than the platform default. Oversized captions read as amateur and cover the frame. Slightly small reads as nonchalant and confident.
• Plain white type with a tight drop shadow, not a heavy outline or a coloured box. The shadow is what keeps it legible over a bright background.
• Centre aligned, at a fixed vertical position. Captions that shift their vertical placement between lines are visually noisy and look broken.
• Two or three words on screen at a time for the first ten seconds, then longer chunks of fifteen to twenty words for the rest. Fast at the start matches the pace of the hook, slower later stops the viewer working too hard.
• Line breaks every four or five words so no line ever wraps unpredictably at phone width.

One more setting that is not styling but ruins more clips than any of the above: encoding quality. A pipeline that re-encodes several times, once for cuts, once for the overlay, once for captions, stacks compression loss at every pass and hands you back something visibly mushy. Specify the output quality explicitly and minimise the number of re-encodes, or your clean edit will look worse than the file you shot.

This is the idea worth taking away even if you never build the pipeline. The machine editing your video sees exactly two things: the waveform and the transcript. That is its entire field of view. It cannot see your face, cannot hear your energy, and does not know what the video is for.

So the test for any editing task is a single question. Is the information this decision needs present in the transcript or the waveform? If yes, automate it and never think about it again. If no, no amount of clever prompting will help, because you are asking for a judgment based on data the model was never given.

Run the test on real tasks and the boundary draws itself:

• Remove a two second gap. In the waveform. Automate it.
• Remove the take where I said the same sentence twice. In the transcript. Automate it.
• Time the captions to the audio. In both. Automate it.
• Pick the better of two clean takes. In neither. Energy and delivery do not survive transcription. Keep it.
• Decide whether this pause is dramatic or dead. In neither. The waveform shows silence and cannot tell you which kind. Keep it.
• Decide if the video is any good. In neither, obviously. Keep it.

This generalises well past editing. Any time you are deciding whether to automate a creative task, ask what the machine can actually see, and be honest about the answer instead of hoping.

A pipeline like this reliably takes a raw file to a publishable short. It does not take a raw file to your best short, and being clear about that gap is what stops you from shipping worse content faster, which is the actual risk here.

What comes back is genuinely finished for a talking head reel: tight, captioned, hooked, correctly sized. For anything with more shape to it, the output is a first assembly and you are still the editor. The things it will not do:

• Choose between takes on delivery. It will keep the last one, which is a rule, not a judgment. If you did four takes and the second was the best, you have to say so.
• Cut for meaning. It removes what is redundant in the transcript, not what is boring. A perfectly fluent thirty seconds that goes nowhere survives every automated pass.
• Add b-roll, cutaways, or anything visual that is not text. There is nothing in the transcript that tells it what an image should be.
• Score it. Music choice and where the track drops are pacing decisions.
• Tell you the hook failed. It will burn a weak hook into the frame just as neatly as a strong one.

The workflow that follows: let the pipeline run overnight on everything, then watch each output once at normal speed before publishing. You are reviewing, not editing, and a review takes about as long as the clip. Twenty hours of scrubbing becomes twenty minutes of watching.

Every one of these showed up on a real first pass. Expect them, and the refinement loop takes one round instead of five.

• The hook overlay renders cropped or cut off, usually because the text is wider than the frame and nothing wrapped it. Specify a maximum character count per line as well as a font size.
• Captions jump around vertically between segments. Pin the vertical position as a fixed value instead of letting it be derived per segment.
• The output is visibly compressed compared to the source, from stacked re-encodes.
• Brand names and technical terms are mistranscribed, and get burned in wrong.
• Punctuation in generated overlay text breaks the render step with a wall of errors.
• Captions are too small or too large because nobody specified a size relative to frame height, so it looks right on your laptop and wrong on a phone.
• The scheduled run reprocesses old files because the move to processed either was not built or ran before the export finished.

The pipeline becomes a system when you stop invoking it. Set a routine to run every morning, check the raw folder for anything new, process what it finds, drop the results in outputs, move the sources to processed. You film into a folder and finished clips appear.

One real decision at this point, and it is not a technical detail. Processing every video at once is fast in wall clock time and extremely heavy on your machine, because encoding will consume every core you have. One at a time takes longer overall and leaves your laptop usable.

• Choose parallel when the run happens while you are away from the desk. Overnight or early morning, before you sit down.
• Choose sequential when you might be working during the run, or when you are on a laptop that throttles under sustained load.
• Choose sequential while you are still refining. Debugging one file at a time is far easier than reading interleaved output from six.

There is one trade-off worth knowing before you rely on this. Locally scheduled routines only run when your computer is awake. A 9am routine on a laptop that is closed at 9am simply does not fire. Either pick a time you know the machine is on and unlocked, or accept that some mornings it will not run and check the outputs folder rather than assuming.

The shortest honest path from reading this to a captioned clip you could actually post. Use one real video you already filmed, not a test clip, because test clips do not contain the retakes and mumbles that expose whether the thing works.

1. 1Make the project folder with raw, outputs, and processed inside it. Drop one real unedited video into raw.
2. 2Open Claude Code in that folder and switch to plan mode before describing anything.
3. 3Describe the pipeline in order: transcribe locally with the free Whisper model, find and cut repeated or blown takes, trim silences, add a hook overlay to the first six seconds, re-transcribe the edited file, burn in styled captions, export to outputs with a dated name.
4. 4Read the plan it writes back. Check specifically that transcription appears twice and that the second pass is on the edited file. If it appears once, say so before approving.
5. 5Approve and let it build. Read the errors as they scroll rather than waiting for it to finish silently.
6. 6Watch the first output all the way through. Write down every problem in plain viewer language: this is cropped, this is jumping, this looks compressed, this word is spelled wrong.
7. 7Hand that whole list back in one message rather than fixing one thing at a time, and let it do a full refinement pass.
8. 8Once two clips in a row come out clean, add the move-to-processed step and set the daily routine.

Two rounds of that is usually the whole build. The first output will be structurally right and cosmetically wrong, which is exactly the correct place to be, because cosmetic problems are the ones you can describe precisely.

The number that matters is not hours saved, it is volume shipped. Fifteen to twenty hours a week spent editing is fifteen to twenty hours not spent making anything new. The win is not better edits, because a machine assembly is slightly worse than a careful manual edit of the same footage. The win is that publishing gets cheap enough that you stop rationing it, and five decent clips a week beats one excellent clip a week by a margin that is not close.

Everything above is the full system and you can build it by hand today with free local tools. Building it yourself is how you learn which settings actually matter, which is why it is written out here rather than hidden behind a download.