How to Give Claude the Ability to Watch Any Video

Claude can't watch video. Anthropic doesn't have a video model yet.

Most tools that try to work around this just pull the transcript and hand it to Claude. That misses half of what's actually in a video, the graphs, the on-screen text, the moment someone holds up a prop or cuts to a chart. A video is two things: pictures and words. Transcript-only tools give Claude one of them.

There's a free skill that gives Claude both, and it's not some obscure hack. The repo sits at 6,300 stars and 740 forks. Here's how it works, what to know before you run it on a long video, six ways to use it, and one workflow that chains it into an actual video-generation pipeline.

The skill uses two tools that have been around for over a decade: yt-dlp downloads the video, ffmpeg turns it into screenshots and pulls the audio. Nothing new or fragile, just two rock-solid pieces of software doing the parts Claude can't do on its own. Claude then reads the frames through its own multimodal Read tool, the same one it uses on any image you'd upload directly.

For the transcript, most videos already have free captions built in, the skill pulls those directly. No cost, no extra step. If a video has no captions, a raw phone recording, a Loom, a reel, the skill falls back to Whisper for transcription.

One thing worth knowing: this isn't Claude-exclusive despite the framing. It's built on the Agent Skills standard, which means the same install works in Codex, Cursor, Copilot, Gemini CLI, and over 50 other agent hosts. If your team runs a mix of tools, this doesn't lock you into one of them.

Three install paths depending on where you work.

Claude Code:

claude.ai (web):

Download watch.skill from the latest release, then drop it into Settings, then Capabilities, then Skills. Turn on "Code execution and file creation" first, the skill shells out to ffmpeg and yt-dlp and won't run without it.

Any other agent host (Codex, Cursor, Copilot, etc):

Everything else installs itself the first time you use it. yt-dlp and ffmpeg set up automatically on macOS through brew, Windows and Linux print the exact commands if you need to run them manually. Once installed, using it is one line:

Paste a link, ask a question if you want one. A 45-minute video gets fully processed in under two minutes, screenshots and transcript both, ready for Claude to reason over.

A Whisper API key through Groq is only needed when a video has no captions. Real cost from testing: five hours of video, fully transcribed, came to $0.39 total.

Groq's free tier gives you two hours of transcription per hour, most people never come close to hitting that limit. For anything with existing captions, which is most public video, the cost is zero either way.

This is the part most people skip past and then wonder why a longer video gets a shallow answer.

The skill caps frame extraction at 2 frames per second and 100 frames total, and best accuracy is under 10 minutes of video. Past that, it prints a "sparse scan" warning, meaning it's sampling the video more thinly to stay under the frame cap, and the analysis gets correspondingly less precise. The fix isn't to avoid long videos, it's to re-run focused on the specific window you actually care about using --start and --end, which is exactly what Use Case 2 below does.

A few other flags worth having in your back pocket:

If you're pulling notes from a slide deck-heavy talk, --resolution 1024 is the difference between Claude guessing at a chart and actually reading the numbers on it.

Found a video doing numbers in your niche? Point Claude at it and ask why.

Claude comes back with the actual mechanics: what the first line claimed, whether the on-screen text matched or contradicted the audio, where the pattern interrupt hit, how many seconds until the first proof point landed.

You don't need Claude to process a 40-minute talk to understand one part of it. Give it a start and end timestamp.

Claude only processes that window. Faster, cheaper, and this is also the fix for the sparse-scan issue above, on anything past 10 minutes, this is how you keep the accuracy up.

Point it at any talk, workshop recording, or webinar and ask for notes instead of a transcript dump.

Because Claude has the screenshots, at full resolution it can reference a specific graph on screen at a specific timestamp, something a transcript-only tool would miss entirely. A 45-minute lecture becomes a page of notes you can actually study from.

If your team records calls in Loom or Zoom, this turns a 40-minute recording into a task list.

This is the one I'd actually build a routine around. Recorded meetings pile up and nobody rewatches them. This gets you the parts that matter without sitting through the parts that don't.

Marketing and sales teams can point this at a competitor's product demo or ad and get the actual structure back.

Because the skill captures screenshots, it catches the on-screen claims and graphics a transcript would miss, which is often where the real persuasion is happening.

This one goes past analysis. Show Claude two or three videos from an editor whose cutting style you like, ask it to identify the pattern, then apply that pattern to your own footage using a video-editing tool like Remotion.

Since Claude has seen the actual frames, not just a description of the edit, it can identify things like exact cut timing and pattern-interrupt placement, then carry that into how it edits or scripts your next video.

Everything above stops at analysis. Claude tells you what worked, you still have to go make something with that information.

There's a second skill that closes that loop: Remotion, an open-source framework maintained by the Remotion team themselves that lets Claude write actual video as React code, then render it to MP4. Instead of a timeline editor, Claude writes components, sequences, and transitions, and the output is a real rendered video, not a description of one.

Paired with /watch, this turns into an input-and-output pipeline. /watch extracts the structure of videos that already work. Remotion builds a new one that follows the same structure.

Install the Remotion skill separately:

This adds a family of Remotion-specific skills Claude Code can call on: /remotion-create for a new composition, /remotion-markup for animation and layout patterns, /remotion-docs to pull current API references while it works.

The actual pipeline looks like this. First, extract the pattern:

Then hand that template straight to Remotion:

Claude has already seen the actual frames of the reference videos, not a summary of them, so the pacing it writes into the Remotion code reflects what it actually observed, not a generic guess at "fast cuts." Render it, preview in Remotion Studio, give correction prompts the same way you'd give feedback to an editor, "slow the second cut," "hold the text overlay half a second longer," and re-render.

This is the version of the guide worth bookmarking. Everything else here tells Claude to look. This tells it to look, then build.

Claude can read and analyze images. It cannot generate them directly, no thumbnail, no photo, no illustration from a text prompt. ChatGPT and Gemini can generate images, Claude can't, and that gap doesn't close.

The Remotion pipeline above gets around this for video specifically, because Remotion output isn't a generated image, it's rendered code, animations, text, shapes, data visualizations. That covers a lot of what people actually need from a video tool. It doesn't cover generating a photorealistic thumbnail or an illustrated frame, that still needs a different tool entirely.

One more thing worth knowing if you're dealing with private or login-gated video: the default setup sends audio to Groq or OpenAI for transcription. If that's a concern, there's a community fork that runs transcription fully on-device with no API key and no network call for the audio at all, worth a look if privacy is the priority over convenience.

Six prompts, one free skill, a two-minute setup, one hard cap worth respecting, and one pipeline that goes from watching to building. That's the whole system.

If this saves you from scrubbing through another 40-minute video looking for the three minutes that mattered, send it to one person still doing that manually.

That's all I'm asking.

Become a paying subscriber to get access to this post and all other premium content.