# Video Frames & Keyframes for AI Vision (Scenes, OCR) (`errandkit/video-frames-for-ai`) Actor

Turn video files into timestamped frames an AI model can see: scene-change keyframes, every N seconds, N evenly spaced or exact timestamps, plus contact sheets and optional OCR of on-screen text. Reads only the parts it needs. No API keys.

- **URL**: https://apify.com/errandkit/video-frames-for-ai.md
- **Developed by:** [Killis Haraway](https://apify.com/errandkit) (community)
- **Categories:** Videos, AI, Developer tools
- **Stats:** 2 total users, 1 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.00 / 1,000 frame extracteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Video Frames & Keyframes for AI Vision (Scenes, OCR)

Turn a video file into a **small set of meaningful still images** that an AI vision model (or a person) can look at: one frame per scene, frames every N seconds, N evenly spaced frames, or frames at exact timestamps. Each frame comes with its **timestamp**, an **image URL**, optional **base64** for agents, optional **OCR text**, and a **contact sheet** that shows the whole video on one image.

- **Scene detection without decoding the whole video.** It reads only keyframes, so a 1-hour video is scanned in seconds.
- **Reads only what it needs.** In interval, count and timestamp modes it fetches just the parts of the file it needs (HTTP range requests), so 10 frames from a 2 GB file don't download 2 GB.
- **Near-duplicate frames are dropped**, so a static slide doesn't come back 30 times.
- **OCR** (Tesseract) reads on-screen text: slides, captions, UI screens, whiteboards.
- Rotation from phone videos is applied. Works with MP4, MOV, WebM, MKV, AVI and most other formats FFmpeg reads.

Built to be called by AI agents: ask for "the key frames of this video", get back JSON plus image URLs.

### What you get

**One item per frame** (`type: "frame"`):

| Field | What it is |
|---|---|
| `timestamp_s`, `timecode` | Position in the video (`00:01:23.400`) |
| `image_url`, `image_key` | The image in the run's key-value store |
| `image_base64` | The image bytes (only with `includeBase64`) |
| `scene_score` | How much the picture changed from the previous keyframe (scenes mode) |
| `ocr_text` | On-screen text (only with `ocr`) |
| `width`, `height`, `content_type`, `video_index`, `video_name`, `frame_index` | |

**One item per video** (`type: "video"`): `status`, `duration_s`, `width`, `height`, `fps`, `codec`, `container`, `rotation`, `has_audio`, `file_bytes`, `bytes_read`, `frames_extracted`, `duplicates_dropped`, `contact_sheets` (`[{key, url}]`), `warnings` and, for failures, `error {code, message}`.

A run summary is saved as `OUTPUT` in the key-value store.

### Quick start

**In the Console:** paste a video URL (or upload a file) and click **Start**.

**API (Python):**

```python
from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("errandkit/video-frames-for-ai").call(run_input={
    "videoUrls": ["https://example.com/demo.mp4"],
    "mode": "scenes",
    "maxFrames": 20,
})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    if item["type"] == "frame":
        print(item["timecode"], item["image_url"])
```

**12 evenly spaced frames with on-screen text** (good for slide decks and screen recordings):

```json
{
  "videoUrls": ["https://example.com/webinar.mp4"],
  "mode": "count",
  "frameCount": 12,
  "ocr": true
}
```

**Exact moments, as base64 for an agent:**

```json
{
  "videoUrls": ["https://example.com/clip.mov"],
  "mode": "timestamps",
  "timestamps": ["0:05", "1:30", "00:12:04.5"],
  "includeBase64": true,
  "imageMaxWidth": 768
}
```

**Example frame item (trimmed):**

```json
{
  "type": "frame",
  "video_name": "demo.mp4",
  "frame_index": 3,
  "timestamp_s": 42.0,
  "timecode": "00:00:42.000",
  "scene_score": 0.31,
  "width": 1024,
  "height": 576,
  "image_key": "v001-f0003-42s.jpg",
  "image_url": "https://api.apify.com/v2/key-value-stores/.../records/v001-f0003-42s.jpg",
  "content_type": "image/jpeg"
}
```

### Use it from an AI agent (MCP)

Works as a tool in Claude, Cursor, ChatGPT connectors and other MCP clients through Apify's MCP server:

```
https://mcp.apify.com/?actors=errandkit/video-frames-for-ai
```

Ask things like "Get the key scenes of this product video and describe each" or "Pull the slides out of this webinar recording and list their titles".

### Input options

| Option | Default | Notes |
|---|---|---|
| `videoUrls` | – | Direct links to video files. Pages on YouTube, TikTok, Instagram, Vimeo etc. are **not** supported |
| `uploadedFile` | – | Upload one video in the Console |
| `videos` | – | API: `[{"name": "a.mp4", "content_base64": "..."}]` for small files (≤ 25 MB) |
| `mode` | `scenes` | `scenes`, `interval`, `count` or `timestamps` |
| `intervalSeconds` | 10 | Interval mode |
| `frameCount` | 12 | Count mode |
| `timestamps` | – | Timestamps mode: seconds or `mm:ss` / `hh:mm:ss.mmm` |
| `maxFrames` | 30 | Per video, all modes (scenes mode keeps the strongest changes) |
| `sceneThreshold` | `"0.12"` | Scenes mode: minimum picture change 0-1. Lower = more frames |
| `dedupe` | `true` | Drop near-identical consecutive frames |
| `imageFormat` | `jpeg` | `jpeg`, `png` or `webp` |
| `imageMaxWidth` | 1024 | 0 = original size |
| `jpegQuality` | 85 | |
| `contactSheet` | `true` | 24 timestamped thumbnails per sheet (free) |
| `includeBase64` | `false` | Image bytes inside each dataset item |
| `ocr` | `false` | Read on-screen text |
| `ocrLanguage` | `eng` | Also `deu`, `fra`, `spa`, `ita`, `por`; combine as `eng+deu` |

### Pricing

Pay per event. Videos that fail are free.

| Event | Price | Per 1,000 |
|---|---|---|
| Frame extracted | $0.001 | $1.00 |
| Frame OCR (only with `ocr`, only frames where text was found) | $0.001 | $1.00 |
| Video data read (per started 10 MB actually downloaded) | $0.001 | $1.00 |
| Actor start (Apify standard) | $0.00005 | $0.05 |

Examples: 12 frames from a 100 MB, 10-minute video in count mode (reads ~25 MB) cost about **$0.015**. Scenes mode reads the whole file: 20 scenes from the same video cost about **$0.03**. Uploaded and base64 files have no data charge. Contact sheets are free. Set *Maximum cost per run* and the Actor stops before going over it, keeping every frame already made.

### Error codes (per video)

| Code | Meaning |
|---|---|
| `PLATFORM_URL_NOT_SUPPORTED` | A YouTube/TikTok/Instagram/Vimeo/... page, not a file. Use a direct file link |
| `UNSUPPORTED_FORMAT` | Not a video (web page, audio only, image) or a file FFmpeg can't open |
| `VIDEO_CORRUPT` | The file opened but its frames couldn't be decoded |
| `DOWNLOAD_FAILED` | The server returned an error or timed out |
| `FILE_TOO_LARGE`, `VIDEO_TOO_LONG` | Over 2 GB or 4 hours |
| `URL_NOT_ALLOWED` | Not http(s), or a private network address |
| `TIMESTAMP_INVALID` | A timestamp couldn't be read or is past the end |
| `MAX_COST_REACHED` | Your maximum cost per run was reached |

### Good to know

- **Scenes mode uses keyframes.** Video encoders put a keyframe at almost every cut, so this finds cuts fast. A video with very few keyframes (some screen recordings) gets a warning: use `interval` or `count` mode for those.
- Frame times in scenes mode are the keyframe times, usually within a fraction of a second of the cut.
- Images are kept with your run's storage (Apify's data retention applies). Image URLs point to your run's key-value store and follow its access settings.

### Privacy

Videos are processed inside your own Apify run. Nothing is sent to any third-party service or AI model, and nothing is kept by the developer.

### Feedback

Something didn't work with your file? Open an issue on the Actor's *Issues* tab with the error code.

Part of the Errandkit catalog of data tools, APIs and MCP servers for AI agents: https://errandkit.com

# Actor input Schema

## `videoUrls` (type: `array`):

Direct links to video files (MP4, MOV, WebM, MKV, AVI, M4V ...). YouTube, TikTok, Instagram and similar page links are not supported.

## `uploadedFile` (type: `string`):

Upload one video file from your computer.

## `videos` (type: `array`):

For API and agent use: \[{"name": "demo.mp4", "url": "https://..."}] or \[{"name": "clip.mp4", "content_base64": "..."}] (base64 up to 25 MB).

## `mode` (type: `string`):

scenes: a frame wherever the picture changes (cuts, new slides). interval: one every N seconds. count: N evenly spaced. timestamps: the exact times you list.

## `intervalSeconds` (type: `integer`):

For mode 'interval'.

## `frameCount` (type: `integer`):

For mode 'count'.

## `timestamps` (type: `array`):

For mode 'timestamps': seconds (83.5) or mm:ss / hh:mm:ss.mmm, one per line.

## `maxFrames` (type: `integer`):

Upper limit in every mode. In scenes mode the strongest changes are kept.

## `sceneThreshold` (type: `string`):

Scenes mode: minimum picture change between keyframes, 0.01-1. Lower finds more scenes. Default 0.12.

## `dedupe` (type: `boolean`):

Skip a frame that looks the same as the previous one (not in timestamps mode). Dropped frames are not charged.

## `imageFormat` (type: `string`):

File type of the frame images.

## `imageMaxWidth` (type: `integer`):

Wider frames are scaled down to this width. 0 keeps the original size. 1024 suits most vision models.

## `jpegQuality` (type: `integer`):

30-100.

## `contactSheet` (type: `boolean`):

Also save contact sheets: up to 24 timestamped thumbnails in one image. Free.

## `includeBase64` (type: `boolean`):

Put each frame's image bytes in its dataset item, for agents that can't open URLs. Use a smaller max width.

## `ocr` (type: `boolean`):

Run OCR on every frame (slides, code, captions). Charged only for frames where text was found.

## `ocrLanguage` (type: `string`):

Tesseract language code(s).

## Actor input object example

```json
{
  "videoUrls": [
    "https://test-videos.co.uk/vids/bigbuckbunny/mp4/h264/360/Big_Buck_Bunny_360_10s_1MB.mp4"
  ],
  "mode": "scenes",
  "intervalSeconds": 10,
  "frameCount": 12,
  "maxFrames": 30,
  "sceneThreshold": "0.12",
  "dedupe": true,
  "imageFormat": "jpeg",
  "imageMaxWidth": 1024,
  "jpegQuality": 85,
  "contactSheet": true,
  "includeBase64": false,
  "ocr": false,
  "ocrLanguage": "eng"
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `summary` (type: `string`):

No description

## `images` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "videoUrls": [
        "https://test-videos.co.uk/vids/bigbuckbunny/mp4/h264/360/Big_Buck_Bunny_360_10s_1MB.mp4"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("errandkit/video-frames-for-ai").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "videoUrls": ["https://test-videos.co.uk/vids/bigbuckbunny/mp4/h264/360/Big_Buck_Bunny_360_10s_1MB.mp4"] }

# Run the Actor and wait for it to finish
run = client.actor("errandkit/video-frames-for-ai").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "videoUrls": [
    "https://test-videos.co.uk/vids/bigbuckbunny/mp4/h264/360/Big_Buck_Bunny_360_10s_1MB.mp4"
  ]
}' |
apify call errandkit/video-frames-for-ai --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,errandkit/video-frames-for-ai"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/4WuPcR2WFm2WQnnbw/builds/Y2nR3zkIkmJg1jY9b/openapi.json