To give an AI agent a YouTube video's transcript and key frames, fetch the captions with yt-dlp or a transcript MCP server, download a 720p copy with yt-dlp, and extract candidate frames with ffmpeg. Then let the agent review the candidates and keep the useful ones. In our single test on one 17-minute video, the transcript arrived in seconds either way. Choosing screenshots took 5 to 7 minutes of agent review.
Both transcript routes returned the same YouTube auto-generated captions, mistakes included. A transcript service saves setup work. It does not give you a better transcript than the captions YouTube already has.
What we tested
We gave a coding agent (Codex CLI, same model and reasoning setting in every group) one job: turn a public YouTube video into a chaptered document with the original transcript text and a set of high-information screenshots. The video was Cole Medin's Your Complete Guide to Fable 5.2, a 17:48 talk with 17 creator-defined chapters and a lot of on-screen slides and code.
We ran three groups:
| Group | Transcript route | Frames | Result |
|---|---|---|---|
| A | yt-dlp downloads the English auto-captions (VTT) | yt-dlp 720p + ffmpeg | Full transcript, 21 screenshots |
| B | Four free transcript websites, fetched by script | yt-dlp 720p + ffmpeg | No transcript (every site returned 403) |
| B2 | ScriptHaul's keyless MCP server (get_transcript), no captions from yt-dlp | yt-dlp 720p + ffmpeg | Full transcript, 20 screenshots |
All numbers on this page come from that one video and one run per group. Treat them as a worked example, not a benchmark.
Group A: yt-dlp captions plus ffmpeg frames
yt-dlp handled metadata, captions, and the video file. The first request for the en caption track returned HTTP 429. Asking for the en-orig track worked on the retry. Caption download took about 6 seconds, the retry about 1 second, and the 720p video about 34 seconds.
ffmpeg then pulled frames at each chapter start plus extra candidates inside dense chapters. Converting the VTT into a clean transcript took under a second of script time. The deduplicated caption stream came to 3,855 words.
The slow step was vision review. The agent spent about 307 seconds (roughly 5 minutes) looking at contact sheets and individual frames before settling on 21 screenshots. The whole run, including installing yt-dlp and ffmpeg into a local virtual environment, took about 9 minutes by the run's own timer.
Group B: transcript websites blocked script requests
Group B tried four free transcript sites: ScriptHaul, BulkTranscripts, Any Video to Text, and SubDownload. The agent fetched each homepage with a plain Python HTTPS request and was told not to bypass protection, log in, or solve challenges.
All four returned HTTP 403. Three saved response bodies read error code: 1010, which is consistent with a Cloudflare access denial. No video was ever submitted and no transcript came back. The agent still produced screenshots from the yt-dlp video, but the document had no spoken text.
This says more about the access route than about the sites. A website built for people in a browser is the wrong interface for an unattended agent. If a tool offers an API or MCP server, use that instead.
Group B2: a transcript MCP server
ScriptHaul publishes a free MCP server at https://scripthaul.com/mcp that needs no account or API key. Its guide says transcripts already in its cache are unlimited, and videos nobody has fetched before count against 20 uncached fetches per connection per day.
We registered the server with the agent and blocked yt-dlp from downloading captions, so the only transcript source was the MCP tool. One get_transcript call returned the timestamped transcript in about 3.8 seconds. The response labeled it as auto-generated English captions. That came to 540 cues and about 3,860 words.
Frame work was the same as in Group A: yt-dlp for a 720p copy, ffmpeg for chapter starts plus scene detection, then vision review. Review took about 410 seconds (roughly 7 minutes) and kept 20 screenshots. The first run took about 11 minutes by its own timer. It skipped full transcript formatting, so we resumed the same session for a short second pass to finish that step.
The transcript was YouTube's auto-captions both times
We compared the transcript words from Group A and Group B2 with a sequence matcher. The two word streams were more than 99.9% identical. They also shared the same recognition errors. For example, CLAUDE.md appeared as "claw.md", and the effort level xhigh appeared as "X high".
So a transcript tool that serves YouTube captions is a delivery convenience. It saves you installing yt-dlp, handling caption formats, and retrying a rate-limited track. It does not fix product names, file names, or jargon. Plan a correction pass for proper nouns and technical terms whichever route you use.
Where the time and tokens went
| Measure (this video, one run each) | Group A | Group B2 |
|---|---|---|
| Transcript fetch | ~6 s + ~1 s retry | ~3.8 s |
| Vision review and frame selection | ~307 s | ~410 s |
| Total run time (run's own timer) | ~9 min | ~11 min, plus a short formatting pass |
| Input tokens (all, mostly cache hits) | ~1.66 million | ~3.56 million (both passes) |
| Uncached input tokens | ~83,000 | ~99,000 (both passes) |
| Screenshots kept | 21 | 20 |
Read the right-hand column with care. Group B2 ran a day later under a different Codex account, and each request carried more context. The B2 agent also chose to add a timestamp re-check pass (about 2 minutes) that Group A did not run. The token and time totals are not a clean head-to-head comparison.
What the table does show is proportion. Getting the text took seconds in both groups. Looking at frames and choosing screenshots took minutes, and that is where most of the agent's work went.
A practical setup for your own agent
- Get the transcript the cheap way. If yt-dlp runs on your machine, ask for the English captions and retry the original-language track (
en-orig) if you hit a 429. If you would rather not maintain yt-dlp, connect a transcript MCP server and check its daily limits first. - Download a small video copy. A 720p render was enough for readable slides and code in this test. Higher resolutions mostly add download time.
- Generate candidates with ffmpeg. Combine chapter starts with scene-change detection. Exact chapter-start frames sometimes show a transition, so keep a few seconds of offsets as backups.
- Let the agent review contact sheets, not single frames. Grids of candidates are faster to judge than dozens of separate images. Expect this step to dominate the run.
- Fix the words that matter. Search the transcript for product names, commands, and file names and correct them against what is on screen.
- Keep a record. Save the selected timestamps, the corrections, and the chapter map next to the output. That processed layer is what you will reuse later.
If you build this as a reusable agent skill, the youtube-clipper skill page and our guide to agent skills for video automation cover how skills package steps like these.
Where caching actually helps
A shared transcript cache, like the one ScriptHaul describes, makes the text instant for anyone who asks for the same video later. In this test the text was already the cheap part. The expensive output was the processed layer: which frames to keep, which caption errors to fix, and how the chapters map to the content.
If you process the same videos repeatedly, or share work across a team, cache those decisions as well as the raw transcript. That is where the minutes went.
Other transcript MCP servers
ScriptHaul is not the only option. Supadata documents an official MCP server with a transcript tool for YouTube and other platforms. It signs in with a Supadata account rather than running keyless. We did not test it in this run, so we make no claims about its speed or output.
Whichever server you pick, test it on your own videos before relying on it. Check long videos, auto-caption-only videos, and non-English videos. And respect the rights of the creator: a transcript is the creator's words, so keep using it for research and notes rather than republishing it.
FAQ
How do I get a YouTube transcript for an AI agent?
Use yt-dlp to download the caption track (for example the English auto-captions), or connect a transcript MCP server so the agent can call a get_transcript tool. In our single test, both routes returned the transcript in seconds.
Is a transcript tool more accurate than yt-dlp captions?
Not in our test. The ScriptHaul MCP transcript and the yt-dlp captions were more than 99.9% identical word for word and shared the same recognition errors. Both were YouTube's auto-generated captions.
Why did transcript websites return 403 to my agent?
In our run, four transcript sites returned HTTP 403 to plain scripted requests, and three response bodies showed Cloudflare error 1010. Use a tool's API or MCP server for agent work instead of scraping its website.
How do I get key frames from a YouTube video?
Download a 720p copy with yt-dlp, extract frames with ffmpeg at chapter starts and at scene changes, and have the agent review contact sheets of the candidates. Expect the review step to take longer than the extraction.
What takes the most time when an AI agent processes a YouTube video?
In our test, choosing screenshots. Fetching the transcript took seconds, while vision review and frame selection took about 5 to 7 minutes per run.
Is the ScriptHaul MCP server free?
Its guide describes a keyless server with unlimited cached transcripts and 20 uncached video fetches per connection per day. Check the current guide before you depend on those limits.

