Open source, v0.1.1, Apache-2.0

Turn any video into context for AI agents

Paste a link or drop a file. Get the transcript, chapters and metadata as Markdown, a .zip, or straight into your agent over MCP.

Try:
  • Free and open source
  • No key needed
  • Runs locally

A real result

One link in, every format out

This is the actual bundle vid2context produces for "Me at the zoo", the first video on YouTube. Every file below was rendered from a recorded run when this site was built.

Me at the zoojawed0:19creator captions

me-at-the-zoo.zip6.9 KB

# Video context pack

This document holds the transcripts and metadata of one video (0:19 in total), prepared by vid2context for an AI model to read. Each transcript paragraph starts with the time it begins in the video, in brackets; linked times open the video at that moment.

Citation rule: when you use something from a video, cite it as [Title @ m:ss](link), using the time and link of the paragraph you drew on (or [Title @ m:ss] when the video has no link).

## Me at the zoo

- Channel: [jawed](https://www.youtube.com/channel/UC4QobU6STFB0P71PMvOGN5A)
- Duration: 0:19
- Source: https://www.youtube.com/watch?v=jNQXAC9IVRw
- Language: en
- Transcript: creator captions from youtube-captions

### Description

> Microplastics are accumulating in human brains at an alarming rate
> https://www.youtube.com/watch?v=0PT5c1z3LL8
>
> “Nanoplastics and Human Health” with Matthew J Campen, PhD, MSPH
> https://www.youtube.com/watch?v=RRBN_4L09Mg
>
> 00:00 Intro
> 00:05 The cool thing
> 00:17 End

### Chapters

- [0:00](https://www.youtube.com/watch?v=jNQXAC9IVRw&t=0s) Intro
- [0:05](https://www.youtube.com/watch?v=jNQXAC9IVRw&t=5s) The cool thing
- [0:17](https://www.youtube.com/watch?v=jNQXAC9IVRw&t=17s) End

### Transcript

#### Intro (0:00)

[0:01](https://www.youtube.com/watch?v=jNQXAC9IVRw&t=1s) All right, so here we are, in front of the elephants

#### The cool thing (0:05)

[0:05](https://www.youtube.com/watch?v=jNQXAC9IVRw&t=5s) the cool thing about these guys is that they have really... really really long trunks and that's cool (baaaaaaaaaaahhh!!) and that's pretty much all there is to say

How it was made

  1. resolveokyoutubeyoutube:jNQXAC9IVRw
  2. metadataokyoutube
  3. captionsokyoutube-captionscreator-captions

Try the live API

This box calls /api/v1/transcript on this site. YouTube often blocks requests from cloud servers, so a video can fail here and still work on your own machine. When it fails, you see the error, the hint and every attempt.

How it works

Captions when they exist, free speech-to-text when they do not

Three steps, the same in the app, the CLI, the HTTP API and the MCP server.

  1. 01

    Paste a link or upload a file

    A link from YouTube, Vimeo, podcast feeds, Google Drive, Dropbox and OneDrive or any other site yt-dlp supports, or an audio or video file from your machine.

  2. 02

    Captions first, then free speech-to-text

    Existing captions are used when the video has them. Otherwise the audio goes through a chain of speech-to-text rungs, local first, then free tiers, then keyless hosts. Every attempt is recorded, so you can see what ran and why.

  3. 03

    Copy it, download it, or hand it to your agent

    Copy Markdown with timestamps, download a .zip with every format, or let Claude, Cursor, VS Code and other agents fetch it themselves over MCP.

The transcript chain

Tried in this order until one works. Each step that runs is recorded with its outcome.

  1. 1
    Creator captions

    Uploaded by the creator, when the video has them.

  2. 2
    Auto-captions

    The platform’s automatic captions, in your language.

  3. 3
    Speech-to-text

    Download the audio and transcribe it through 13 Whisper-class rungs.

Sources

Works where your videos are

Platform providers handle the big sites directly. Everything else goes through the yt-dlp provider.

  • YouTubeYouTube provider
  • VimeoVimeo provider
  • TikTokwith the yt-dlp provider
  • Xwith the yt-dlp provider
  • Instagramwith the yt-dlp provider
  • Twitchwith the yt-dlp provider
  • PodcastsDirect media URL provider
  • Google Drive, Dropbox and OneDrive linksDirect media URL provider
  • Local filesLocal media file provider

Free first

Bring any model

67 rungs across 33 services, tried in free-first order. A rung is one provider, one model and one credential. When one fails, rate-limits or is down, the chain moves to the next.

Local

16 rungs

Runs on your machine. Free, private, nothing leaves it.

Speech-to-text

  • localLOCAL_STT_BASE_URL
  • transformersonnx-community/whisper-base
  • whisper-cppWHISPER_CPP_MODEL

LLM

  • ollamaOLLAMA_MODEL
  • lmstudioLMSTUDIO_MODEL
  • llamacppLLAMACPP_MODEL
  • vllmVLLM_MODEL
  • janJAN_MODEL
  • localaiLOCALAI_MODEL
  • localLOCAL_LLM_MODEL

Embeddings

  • ollamaOLLAMA_EMBED_MODEL
  • lmstudioLMSTUDIO_EMBED_MODEL
  • localLOCAL_EMBED_MODEL
  • transformersXenova/multilingual-e5-small

Vision

  • ollamaOLLAMA_VISION_MODEL
  • tesseracteng

Free tier

32 rungs

Hosted free tiers that use your own key.

Speech-to-text

  • cloudflare-binding@cf/openai/whisper-large-v3-turbo
  • cloudflare-binding@cf/openai/whisper
  • cloudflare@cf/openai/whisper-large-v3-turbo
  • cloudflare@cf/openai/whisper
  • groqwhisper-large-v3-turbo
  • mistralvoxtral-mini-transcribe-3

LLM

  • cloudflare-binding@cf/meta/llama-3.3-70b-instruct-fp8-fast
  • cloudflare-binding@cf/openai/gpt-oss-120b
  • cloudflare-binding@cf/meta/llama-4-scout-17b-16e-instruct
  • cloudflare-binding@cf/mistralai/mistral-small-3.1-24b-instruct
  • nvidianvidia/nemotron-3-super-120b-a12b
  • sambanovaMeta-Llama-3.3-70B-Instruct
  • cloudflare@cf/meta/llama-3.3-70b-instruct-fp8-fast
  • zhipuglm-4.7-flash
  • siliconflowQwen/Qwen3-8B
  • gemini-nativegemini-3.8-flash
  • gemini-nativegemini-3.5-flash-lite
  • groqqwen/qwen3.8-27b
  • groqopenai/gpt-oss-120b
  • groqopenai/gpt-oss-20b
  • cerebrasgpt-oss-120b
  • cerebrasllama-3.3-70b
  • mistralmistral-small-latest
  • githubmeta/Llama-3.3-70B-Instruct
  • geminigemini-2.5-flash-lite
  • openrouterfree-auto

Embeddings

  • cloudflare-binding@cf/baai/bge-m3
  • geminigemini-embedding-2
  • jinajina-embeddings-v3
  • cloudflare@cf/baai/bge-m3
  • mistralmistral-embed

Vision

  • geminigemini-2.5-flash-lite

Keyless

10 rungs

Hosted and free with no account at all.

Speech-to-text

  • ovhwhisper-large-v3-turbo

LLM

  • kilonvidia/nemotron-3-ultra-550b-a55b:free
  • kilonvidia/nemotron-3.5-super-vl:free
  • kilonvidia/nemotron-3.5-lightning:free
  • ovhMeta-Llama-3_3-70B-Instruct
  • ovhgpt-oss-120b
  • pollinationsopenai-fast

Embeddings

  • ovhbge-m3
  • ovhQwen3-Embedding-8B

Vision

  • ovhQwen2.5-VL-72B-Instruct

Paid

9 rungs

Last backstop, used only when you set its key.

Speech-to-text

  • deepinfraopenai/whisper-large-v3-turbo
  • togetheropenai/whisper-large-v3
  • openaiwhisper-1

LLM

  • anthropicclaude-haiku-5-5
  • anthropicclaude-sonnet-5-5
  • vertex-geminigemini-3.8-flash
  • vertex-anthropicclaude-haiku-5-5
  • openaigpt-5.4-nano

Embeddings

  • openaitext-embedding-3-small

Every model id can be overridden from the environment, and extra free keys can be pooled into their own rungs. Limits and env vars for each rung are in Providers and rungs.

Agents and MCP

Your agent fetches the video itself

Add the MCP server once (vid2context-mcp on npm) and ask about any video. No key needed.

Install in VS CodeVS Code InsidersAdd to CursorAdd to LM StudioAdd to Goose

Claude Code

Guide

One command, user scope, every project.

claude mcp add --scope user --transport stdio vid2context -- npx -y vid2context-mcp

Claude Desktop

Guide

Add one block to claude_desktop_config.json.

{
  "mcpServers": {
    "vid2context": {
      "command": "npx",
      "args": [
        "-y",
        "vid2context-mcp"
      ]
    }
  }
}

Cursor

Guide

One-click install, or ~/.cursor/mcp.json.

{
  "mcpServers": {
    "vid2context": {
      "command": "npx",
      "args": [
        "-y",
        "vid2context-mcp"
      ]
    }
  }
}

VS Code

Guide

One-click install for GitHub Copilot agent mode.

{
  "servers": {
    "vid2context": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "-y",
        "vid2context-mcp"
      ]
    }
  }
}

ChatGPT

Guide

Upload a context pack to a Project or custom GPT. ChatGPT connects only to remote MCP servers over HTTPS.

npx -y vid2context pack "https://www.youtube.com/watch?v=jNQXAC9IVRw" > context.md

Also: Codex, Gemini CLI, Windsurf, Cline, LM Studio, Goose.

28 tools your agent can call

Listed by the MCP server in the repo when this site was built.

  • add_to_collectionIngest videos into a collection now: each URL is a video or a list (playlist, channel, feed), and lists are expanded.
  • askAnswer a question from the transcripts of one or more videos, or a saved collection, with server-side retrieval and the free-first LLM chain.
  • build_briefBuild a cited brief across videos on the server with a free LLM (map-reduce for large collections).
  • choose_caption_trackList a video's caption tracks (languages, creator or auto-generated) and ask the user to pick one, then return that transcript with [m:ss] timestamps.
  • create_collectionCreate a research collection.
  • digest_videosBuild one cited digest of several videos (a playlist, a collection, or a list of URLs) using MCP sampling: the server asks the client's model to take notes on each part of each transcript, then combine them.
  • export_context_packOne Markdown document for many videos (the pack format): a header per video with title, channel, URL and chapters, then each transcript in timestamped paragraphs.
  • export_notionCreate one Notion page per video under a parent page (chapters as headings, paragraphs with timestamp links, a bookmark to the video).
  • export_obsidianWrite one Markdown note per video into a folder of an Obsidian vault on this machine: frontmatter properties, chapters as headings, paragraphs prefixed with [m:ss] timestamp links, plus an index note with [[wikilinks]] when a collection is exported.
  • extractExtract structured JSON from videos with a template (buyer-guide, tutorial-steps, comparison, summary, brief) or your own JSON Schema.
  • find_duplicatesCompare the transcripts of several videos and group reuploads (the same video uploaded again) and clips (an excerpt of a longer video, with the [m:ss] where it starts in the original and the range it covers).
  • get_collectionA collection with a summary of each video (id, title, channel, duration, URL, transcript origin), the sources it follows, inputs waiting for the next refresh and its watch list.
  • get_commentsTop comments of a video, with creator replies and pinned comments flagged.
  • get_key_momentsThe moments of a video worth jumping to, picked by the free-first LLM chain from the timestamped transcript.
  • get_metadataTitle, channel, description, duration, publish date, chapters, tags and language for a video URL or local file, without fetching the transcript.
  • get_outlineStart here for any video longer than about 20 minutes.
  • get_sectionOne chapter or time range of a video's transcript, with the same header as get_transcript.
  • get_skip_segmentsCommunity-marked SponsorBlock segments of a YouTube video: sponsor reads, self-promotion, subscribe reminders, intros, outros, previews and filler, each with its [m:ss] range, a timestamp URL and how much of the transcript it covers (lines, word share and the first words).
  • get_transcriptThe full transcript of one video or local media file, with a header (title, channel, URL, duration, how the transcript was made, chapters).
  • list_caption_languagesList the caption tracks a video has (language, name, creator or auto-generated) and, for YouTube, the languages YouTube can machine-translate them into.
  • list_collectionsList research collections with how many videos and sources each has.
  • list_formatsThe output formats get_transcript and get_section accept (text, json, markdown, srt, vtt, pack, ...), with what each is for.
  • list_itemsExpand a playlist, channel, podcast feed or page into its videos (a single video URL returns itself).
  • list_modelsThe models (rungs) this server can use for speech-to-text, LLM, embedding, video and visual work, in free-first order: tier (local, free, keyless, paid), quality 1 to 5, capabilities (json, tools, vision, stream, ...), context window, whether it is enabled, and the names of the env vars that would enable it.
  • open_transcript_viewerShow a video transcript in an interactive viewer (chapters, search, clickable [m:ss] timestamps, one-click citations, send selected lines to the chat) in clients that support MCP Apps, such as ChatGPT and Claude.
  • research_videosSearch 1 to 10 videos (or a playlist, capped at 10) for a question and get only the relevant excerpts: ranked by BM25, each with the surrounding context, [m:ss] and a timestamp link, under a token cap.
  • search_transcriptFind where something is said across saved video transcripts, instead of reading whole transcripts (about 12k tokens per hour).
  • watch_liveTranscribe a live stream (a live YouTube video or an HLS .m3u8 URL) while it runs.

Use cases

Hours of video, read in seconds

Research a buying decision across a playlist

A buyer’s series on the Mercedes G-Wagon spans several long episodes. Load the episodes into your agent and ask which model, year and mileage to buy. Every transcript line carries its timestamp, so each claim in the answer can cite the episode and the moment that said it.

get_transcript
Title: Best & Worst G-Wagon Years: Reliability, Interiors & Engines Ranked!
Channel: Alliance Motorworks
URL: https://www.youtube.com/watch?v=cCPsdFdlshc
Duration: 18:04
Transcript: auto-captions, en (youtube-captions)

[0:00] so you're going to buy G wagon which
[0:02] year is best for you well in this video
[0:03] we're going to do a year-by-year
[0:04] breakdown of the legendary
[0:08] w463 platform so you can check out the
[0:10] model year ranges the engine options and
[0:13] the interior because if you buy the
[0:16] wrong one I don't want you getting
[0:18] buyers remorse or ending up with a bad
[0:21] specimen my name is Adam welcome to the
[0:23] U uh gwh and YouTube channel I do
[0:25] everything here for the love of G I'm
The first lines of "Best & Worst G-Wagon Years: Reliability, Interiors & Engines Ranked!" (18:04, 521 caption lines), as the agent receives them

Learn from tutorials

Chapters become sections. For a long course video, your agent reads the outline first and pulls only the section it needs, instead of the whole transcript.

Notes from podcasts

Podcast episodes have no captions, so the audio goes through speech-to-text. You get Markdown notes with a timestamp on every paragraph, ready for Obsidian, Notion or your agent.

Self-host

Run it on your own machine or your own cloud

The same TypeScript core runs everywhere. Locally it has the best success rate, because requests come from your own connection.

Docker

Compose file for the full server (HTTP API, yt-dlp, ffmpeg), plus a small stdio MCP image at the repo root.

docker compose -f deploy/containers/compose.yaml up -d --build
Docker guide

Cloudflare

One Worker serves the site, the apps and the API, with Workers AI as an optional free rung.

npm run deploy
Cloudflare guide

Vercel

A prebuilt Build Output tree: the same layout with the API as a function.

npm run build:vercel
Vercel guide

GitHub Pages

A client-only build: the apps run in the browser with your own keys or local models.

npm run deploy:pages
GitHub Pages guide

FAQ

Questions people ask

What does it cost?

Nothing. vid2context is free and open source under the Apache-2.0 license. With no keys it uses captions and keyless hosts. Paid services run only when you set their key, and VID2CONTEXT_ALLOW_PAID=0 turns them off entirely.

Do I need an API key?

No. Captions need no key, and speech-to-text falls back to keyless Whisper hosts. Free keys (Groq, Gemini, Mistral and others) unlock 32 more free-tier rungs that are faster and have higher limits.

Where does my data go?

When you run vid2context yourself, captions are fetched straight from the platform and your files stay on your machine unless a hosted rung transcribes them. Keyless rungs send audio to third-party hosts without an account. Set VID2CONTEXT_POLICY=local-only to keep everything on your machine, or VID2CONTEXT_KEYLESS=0 to skip keyless hosts.

Which sites work?

YouTube, Vimeo, TikTok, X, Instagram, Twitch, Podcasts, Google Drive, Dropbox and OneDrive links, Local files.

Why does a YouTube video fail on the hosted demo?

YouTube often blocks requests from cloud servers, so the demo on this site can fail for videos that work fine on your own machine. Run the CLI or the MCP server locally, where requests come from your own connection.

Which agents does it work with?

Any MCP client: Claude Code, Claude Desktop, Cursor, VS Code, Codex, Gemini CLI, Windsurf, Cline, LM Studio and Goose. ChatGPT gets a context pack file you upload to a Project.