Video Frames & Keyframes for AI Vision (Scenes, OCR)

Turn a video file into a small set of meaningful still images that an AI vision model (or a person) can look at: one frame per scene, frames every N seconds, N evenly spaced frames, or frames at exact timestamps. Each frame comes with its timestamp, an image URL, optional base64 for agents, optional OCR text, and a contact sheet that shows the whole video on one image.

• Scene detection without decoding the whole video. It reads only keyframes, so a 1-hour video is scanned in seconds.
• Reads only what it needs. In interval, count and timestamp modes it fetches just the parts of the file it needs (HTTP range requests), so 10 frames from a 2 GB file don't download 2 GB.
• Near-duplicate frames are dropped, so a static slide doesn't come back 30 times.
• OCR (Tesseract) reads on-screen text: slides, captions, UI screens, whiteboards.
• Rotation from phone videos is applied. Works with MP4, MOV, WebM, MKV, AVI and most other formats FFmpeg reads.

Built to be called by AI agents: ask for "the key frames of this video", get back JSON plus image URLs.

What you get

One item per frame (type: "frame"):

Field	What it is
timestamp_s, timecode	Position in the video (00:01:23.400)
image_url, image_key	The image in the run's key-value store
image_base64	The image bytes (only with includeBase64)
scene_score	How much the picture changed from the previous keyframe (scenes mode)
ocr_text	On-screen text (only with ocr)
width, height, content_type, video_index, video_name, frame_index

One item per video (type: "video"): status, duration_s, width, height, fps, codec, container, rotation, has_audio, file_bytes, bytes_read, frames_extracted, duplicates_dropped, contact_sheets ([{key, url}]), warnings and, for failures, error {code, message}.

A run summary is saved as OUTPUT in the key-value store.

Quick start

In the Console: paste a video URL (or upload a file) and click Start.

API (Python):

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("errandkit/video-frames-for-ai").call(run_input={
    "videoUrls": ["https://example.com/demo.mp4"],
    "mode": "scenes",
    "maxFrames": 20,
})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    if item["type"] == "frame":
        print(item["timecode"], item["image_url"])

12 evenly spaced frames with on-screen text (good for slide decks and screen recordings):

{
  "videoUrls": ["https://example.com/webinar.mp4"],
  "mode": "count",
  "frameCount": 12,
  "ocr": true
}

Exact moments, as base64 for an agent:

{
  "videoUrls": ["https://example.com/clip.mov"],
  "mode": "timestamps",
  "timestamps": ["0:05", "1:30", "00:12:04.5"],
  "includeBase64": true,
  "imageMaxWidth": 768
}

Example frame item (trimmed):

{
  "type": "frame",
  "video_name": "demo.mp4",
  "frame_index": 3,
  "timestamp_s": 42.0,
  "timecode": "00:00:42.000",
  "scene_score": 0.31,
  "width": 1024,
  "height": 576,
  "image_key": "v001-f0003-42s.jpg",
  "image_url": "https://api.apify.com/v2/key-value-stores/.../records/v001-f0003-42s.jpg",
  "content_type": "image/jpeg"
}

Use it from an AI agent (MCP)

Works as a tool in Claude, Cursor, ChatGPT connectors and other MCP clients through Apify's MCP server:

https://mcp.apify.com/?actors=errandkit/video-frames-for-ai

Ask things like "Get the key scenes of this product video and describe each" or "Pull the slides out of this webinar recording and list their titles".

Input options

Option	Default	Notes
videoUrls	–	Direct links to video files. Pages on YouTube, TikTok, Instagram, Vimeo etc. are not supported
uploadedFile	–	Upload one video in the Console
videos	–	API: [{"name": "a.mp4", "content_base64": "..."}] for small files (≤ 25 MB)
mode	scenes	scenes, interval, count or timestamps
intervalSeconds	10	Interval mode
frameCount	12	Count mode
timestamps	–	Timestamps mode: seconds or mm:ss / hh:mm:ss.mmm
maxFrames	30	Per video, all modes (scenes mode keeps the strongest changes)
sceneThreshold	"0.12"	Scenes mode: minimum picture change 0-1. Lower = more frames
dedupe	true	Drop near-identical consecutive frames
imageFormat	jpeg	jpeg, png or webp
imageMaxWidth	1024	0 = original size
jpegQuality	85
contactSheet	true	24 timestamped thumbnails per sheet (free)
includeBase64	false	Image bytes inside each dataset item
ocr	false	Read on-screen text
ocrLanguage	eng	Also deu, fra, spa, ita, por; combine as eng+deu

Pricing

Pay per event. Videos that fail are free.

Event	Price	Per 1,000
Frame extracted	$0.001	$1.00
Frame OCR (only with ocr, only frames where text was found)	$0.001	$1.00
Video data read (per started 10 MB actually downloaded)	$0.001	$1.00
Actor start (Apify standard)	$0.00005	$0.05

Examples: 12 frames from a 100 MB, 10-minute video in count mode (reads ~25 MB) cost about $0.015. Scenes mode reads the whole file: 20 scenes from the same video cost about $0.03. Uploaded and base64 files have no data charge. Contact sheets are free. Set Maximum cost per run and the Actor stops before going over it, keeping every frame already made.

Error codes (per video)

Code	Meaning
PLATFORM_URL_NOT_SUPPORTED	A YouTube/TikTok/Instagram/Vimeo/... page, not a file. Use a direct file link
UNSUPPORTED_FORMAT	Not a video (web page, audio only, image) or a file FFmpeg can't open
VIDEO_CORRUPT	The file opened but its frames couldn't be decoded
DOWNLOAD_FAILED	The server returned an error or timed out
FILE_TOO_LARGE, VIDEO_TOO_LONG	Over 2 GB or 4 hours
URL_NOT_ALLOWED	Not http(s), or a private network address
TIMESTAMP_INVALID	A timestamp couldn't be read or is past the end
MAX_COST_REACHED	Your maximum cost per run was reached

Good to know

• Scenes mode uses keyframes. Video encoders put a keyframe at almost every cut, so this finds cuts fast. A video with very few keyframes (some screen recordings) gets a warning: use interval or count mode for those.
• Frame times in scenes mode are the keyframe times, usually within a fraction of a second of the cut.
• Images are kept with your run's storage (Apify's data retention applies). Image URLs point to your run's key-value store and follow its access settings.

Privacy

Videos are processed inside your own Apify run. Nothing is sent to any third-party service or AI model, and nothing is kept by the developer.

Feedback

Something didn't work with your file? Open an issue on the Actor's Issues tab with the error code.

Part of the Errandkit catalog of data tools, APIs and MCP servers for AI agents: https://errandkit.com