Video frames and transcripts
Large language models cannot directly watch video, but they can read images and text. claude-video converts a video into these two inputs: frames and timestamped transcripts, which Claude processes in parallel. Brad Bonanno created the /watch command; it is packaged as a Claude Skill, using yt-dlp for downloads, ffmpeg for frame extraction and Whisper for missing subtitles.
The process has seven steps: provide a video URL or local path and a question; yt-dlp first tries native subtitles, which are free and immediately available; ffmpeg selects scene-change frames or keyframes according to the detail mode; if subtitles are unavailable, Whisper API transcribes the audio, with Groq preferred and OpenAI as fallback; frames and timestamped subtitles go to Claude for parallel image reading; Claude answers from actual visual and audio content rather than metadata; finally, the working directory is printed and cleaned up if there are no follow-up questions.
It supports YouTube, Loom, TikTok, X, Instagram, Vimeo and 100+ platforms through yt-dlp. Local files can be .mp4, .mov, .mkv or .webm. Typical uses include timestamp questions, hour-long lecture summaries and locating UI failures in screen recordings.
URL / file
yt-dlp download
ffmpeg frames
Subtitles / Whisper
Claude image reading
Content-based answers
Claude answers based on the actual visual and audio content of the video — not its metadata.
Four installation options
The dependencies are ffmpeg and yt-dlp; macOS installs them automatically with brew, while Linux / Windows prints the commands for you to run. Whisper transcription is optional: most public videos have free native subtitles, and Whisper is used only when subtitles cannot be retrieved, requiring a Groq (recommended) or OpenAI API key. The first run creates a key template at ~/.config/watch/.env.
Claude Code (recommended)
# Add the marketplace, then install the watch plugin
/plugin marketplace add bradautomates/claude-video
/plugin install watch@claude-videoOther agents: Codex, Cursor, Copilot, Gemini CLI and 50+ hosts
Use the skills CLI for a one-line global installation on more than 50 AI coding hosts:
npx skills add bradautomates/claude-video -gManual / developer installation
Clone the repository and symlink skills/watch into Claude's skills directory:
git clone https://github.com/bradautomates/claude-video.git
ln -s "$(pwd)/claude-video/skills/watch" ~/.claude/skills/watchDetail modes and token costs
The --detail flag in /watch determines frame counts and the extraction engine, affecting speed and image tokens. The figures below were measured on the same 49-minute 08-second video. Other flags control sampling, transcription and budgets more precisely; the default balanced mode suits most videos, with other flags used for a segment of a long video, screen text or full coverage.
Mode · 01
--detail transcript
Subtitles only
Uses subtitles and 0 frames, taking about 4.5s with 0 image tokens. The lowest-cost option for text-oriented videos such as lectures and podcasts.
Mode · 02
--detail efficient
Keyframes
Extracts keyframes, capped at 50 frames, taking about 0.5s and about 9.8k tokens. Use for low-motion content or speed.
Mode · 03
--detail balanced
Scene detection
The default. Scene-change extraction, capped at 100 frames, taking about 20.9s and about 19.7k tokens. Balanced coverage.
Mode · 04
--detail token-burner
Full coverage
Uncapped scene-change extraction, taking about 21s and about 22.8k tokens. Use for high-motion videos or full sampling.
Sampling · 05
--start / --end
Time window
Analyzes only the specified segment, such as --start 2:15 --end 2:45. Denser sampling within the budget and fewer tokens.
Sampling · 06
--timestamps T1,T2,…
Timestamp frames
Extracts one frame at each absolute timestamp, adding it to the detail budget. Use for specific frames.
Sampling · 07
--resolution W
Frame resolution
Defaults to 512px; use 1024 for small screen text, which also increases tokens.
Transcription · 08
--whisper groq|openai
Transcription backend
Selects the Whisper backend; Groq has lower cost and higher speed, with OpenAI as fallback.
Transcription · 09
--no-whisper
Frames only
Disables transcription and sends only frames. Use when you do not need a transcript or want lower cost.
Budget · 10
--max-frames / --no-dedup
Limits and deduplication
Near-duplicate frames are removed by default; --max-frames N lowers the cap, and --no-dedup keeps all frames.
Content and mode selection
| Content to analyze | Recommended mode / flags | Reason |
|---|---|---|
| Lectures, podcasts and speech-focused content | --detail transcript | Subtitles only, 0 frames; fastest and cheapest. |
| Low-motion content and quick scans | --detail efficient | Keyframes, about 0.5s and the lowest token use. |
| General video (default case) | --detail balanced | Scene detection balances coverage and cost. |
| High-motion content with full coverage | --detail token-burner | Uncapped, with the fullest coverage and highest token use. |
| A segment of a long video | --start / --end | Focuses the budget on the segment and avoids sparse scans. |
Six settings for cost and precision
These six settings come from the official README and control /watch analysis precision and token cost.
Mechanism 01 · Subtitles first, Whisper fallback
yt-dlp first tries native subtitles, which are free and immediately available for most public videos. Whisper API transcription runs only when subtitles cannot be retrieved. Most cases do not need an API key.
Source · README · How It Works
Mechanism 02 · Duration and frame budgets
The default budget is calculated from duration: ≤30 seconds gives about 30 frames, 1–3 minutes about 60, and 3–10 minutes about 80. Videos longer than 10 minutes are capped at 100 frames and produce a sparse scan message.
Source · README · Frame Budget
Mechanism 03 · Resolution for small screen text
Frames default to 512px wide. For UI text, code or subtitles, increase it with --resolution 1024; image tokens increase substantially, so avoid applying it unnecessarily to an entire long video.
Source · README · Key Flags
Mechanism 04 · Automatic near-duplicate removal
Deduplication is on by default: each frame becomes a 16×16 grayscale thumbnail and is compared with the previous retained frame using mean absolute difference; a difference ≤2.0 discards it. The budget applies after deduplication. Use --no-dedup to retain every frame.
Source · README · Frame Deduplication
Mechanism 05 · Groq as preferred Whisper backend
When transcription is needed, Groq is preferred for cost and speed; OpenAI is an alternative. Select it with --whisper groq|openai and put keys in ~/.config/watch/.env, created on the first run.
Source · README · Dependencies & Keys
Mechanism 06 · Frame counts dominate image tokens
Transcript tokens are relatively cheap; frames account for most of the cost. Tokens increase with frame count and resolution, so reserve token-burner and --resolution 1024 for cases requiring that precision.
Source · README · Detail Modes
UI diagnosis with /watch
Example scenario: a checkout button becomes gray and unclickable in a screen recording, but you do not remember which action triggered it. Send the video and question to /watch to extract frames, read subtitles and identify timestamps. The long-video example uses --start / --end to focus on a segment.
$ You › /watch ~/Movies/checkout-bug.mov At what second does the checkout button become unclickable?
claude: Claude ›
hl: The button turns gray at 00:41.
The sequence is:
· 00:38 You change the quantity from 1 to 0
· 00:41 The subtotal becomes $0.00,and "Checkout" becomes disabled
· 00:44 Clicking the button does nothing because it is already disabled
At 00:42, the narration says "Why can't I click it?", matching the screen.
The root cause is that quantity 0 is accepted and sets the subtotal to zero, not the button itself.
$ You › /watch https://youtu.be/<id> --start 12:30 --end 13:15 What commands does this segment demonstrate?
claude: Claude ›
These 45 seconds demonstrate terminal operations:the speaker first runs git status,
then pastes some npm run build output at 12:47,
a red error appears at 13:02; he changes one configuration line and reruns successfully.
Give Claude the ability to watch any video.
Frame, subtitle and timestamp checks
/watch converts video into two inputs Claude can read natively: frames and timestamped subtitles. Its answers use actual frames rather than guesses from filenames or descriptions. Timestamps let it identify "turned gray at 00:41", while subtitles and frames provide cross-checks.
Both examples also control cost: the local file uses the default balanced mode, and deduplication removes two-thirds of duplicate frames; the long video uses --start / --end for only the relevant 45 seconds, giving denser sampling and fewer tokens. Add --resolution 1024 when text is unreadable, without increasing resolution for the entire video.
Usage limits
Advanced /watch settings
Basic usage is /watch <URL 或路徑> <你的問題>, which runs the default balanced mode without flags. Combine flags according to the analysis needed.
Advanced settings
1. Control long-video cost with time windows. Use --start / --end for relevant segments instead of the entire video, such as --start 1:12:00. Sampling is denser, tokens are lower and sparse scans are avoided.
2. Extract frames at specified timestamps. If you know which timestamps matter, use --timestamps for one frame at each absolute time, added to the detail budget, without relying on scene detection.
3. Increase resolution for screen recordings. Use --resolution 1024 (default 512) to read UI text, code or subtitles verbatim. Apply it only when needed to avoid spending tokens on the entire video.
4. Choose the transcription backend. Use --whisper groq for cost and speed, with --whisper openai as fallback; use --no-whisper for frames only when no transcript is needed.
5. Install in your usual agent. Besides Claude Code, npx skills add bradautomates/claude-video -g installs in Codex, Cursor, Copilot, Gemini CLI and 50+ hosts, so the workflow can stay in the same tool.
Three further links
① github.com/bradautomates/claude-video: Official README with flags, detail modes and dependencies. ② YouTube · @bradbonanno: Creator Brad Bonanno's channel, demonstrations and background. ③ solarisautomation.io: Solaris Automation, with which the creator is affiliated.
Built on yt-dlp, ffmpeg, Claude's multimodal Read tool, and Whisper via Groq or OpenAI.