# Speech to Text (Whisper): Video & Audio Transcription (`andrew_babo/speech-to-text-whisper`) Actor

Word-level speech to text on CPU with whisper.cpp. Transcribe video or audio in 90+ languages and get word timestamps, confidence scores and ready-to-use caption segments as JSON.

- **URL**: https://apify.com/andrew\_babo/speech-to-text-whisper.md
- **Developed by:** [Andrew Babo](https://apify.com/andrew_babo) (community)
- **Categories:** AI, Videos
- **Stats:** 26 total users, 22 monthly users, 49.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Speech to Text (Whisper) — Video & Audio Transcription with Word Timestamps

Turn any audio or video file into an accurate transcript with **word-level
timestamps**, confidence scores and ready-to-use caption segments. Runs
whisper.cpp on CPU — no GPU, no API keys, no local install.

**Use it for:** subtitles and SRT-style captions, searchable video archives,
podcast show notes, meeting notes, content repurposing, dataset labelling.

- 90+ languages, or automatic language detection
- Word timestamps, so captions can be split to any length
- Accepts MP4, MOV, MKV, WebM, MP3, WAV, M4A, OGG… (converted internally to 16 kHz mono)
- Accepts a public URL or a key-value-store record from a previous run
- Optional time window, so long media can be sharded across parallel runs

### Quick start

```json
{
  "op": "transcribe",
  "source": "https://example.com/interview.mp4",
  "options": { "preset": "balanced", "language": "en" }
}
```

Feature-detect the build (free, a few seconds, needs no source):

```json
{ "op": "capabilities" }
```

### Input

| Field | Type | Notes |
| --- | --- | --- |
| `op` | `transcribe` | `capabilities` | default `transcribe` |
| `source` | string | `https://` URL or `kv:<storeId>/<key>`. Audio or video. |
| `options.preset` | `fast` | `adaptive` | `balanced` | `accurate` | speed vs accuracy |
| `options.language` | ISO code (`en`, `vi`, `es`…) | omit for auto-detect |
| `options.model` | model name | overrides the preset default |
| `options.threads` | integer | defaults to the run's vCPU count |
| `options.segment_gap_ms` | number | silence gap that starts a new caption segment |
| `options.start_sec` / `duration_sec` / `overlap_sec` | number | transcribe one window only |
| `options.keep_raw_json` | boolean | also store the raw whisper output |
| `output.signed_upload_url` | string | PUT `transcript.json` into your own storage |
| `cleanup` | `on_success` | `always` | `off` | artifact retention |
| `callback` | object | `{ url, secret_header: { name, value } }` webhook |

**Presets**

| Preset | Best for | Relative speed |
| --- | --- | --- |
| `fast` | drafts, search indexing | fastest |
| `adaptive` | mixed content, unknown quality | fast |
| `balanced` | default choice for captions | medium |
| `accurate` | publishing, noisy audio, accents | slowest |

### Output

```json
{
  "status": "success",
  "op": "transcribe",
  "artifacts": [{ "name": "transcript", "kv_key": "transcript.json", "url": "https://api.apify.com/v2/key-value-stores/.../transcript.json" }],
  "meta": {
    "language": "en",
    "duration_sec": 338.4,
    "word_count": 912,
    "model": "…",
    "words": [
      { "id": 0, "text": "Hello", "startMs": 120, "endMs": 410, "confidence": 0.98 }
    ],
    "segments": [
      { "startMs": 120, "endMs": 4120, "text": "Hello and welcome back to the show" }
    ]
  }
}
```

- `words[]` — one entry per word with start/end in milliseconds and confidence.
- `segments[]` — sentence-like caption blocks, split on pauses (`segment_gap_ms`).
- The full transcript is also stored as the `transcript.json` artifact.

#### Sharding a long file

Pass a window and the actor transcribes only that slice; timestamps are offset
back to the original timeline, and words inside the tail overlap are flagged
`_overlap` so you can de-duplicate when merging shards.

```json
{
  "op": "transcribe",
  "source": "https://example.com/audio16k.wav",
  "options": { "start_sec": 600, "duration_sec": 300, "overlap_sec": 1.5 }
}
```

For heavy parallel fan-out, use the dedicated **Fast Speech to Text** actor,
which keeps the model warm and processes a list of windows per run.

### Error handling

```json
{ "ok": false, "reason": "BAD_INPUT", "message": "..." }
```

| `reason` | Meaning | What to do |
| --- | --- | --- |
| `BAD_INPUT` | missing/unreadable `source` | verify the URL is publicly reachable |
| `UPSTREAM_BLOCKED` | the host refused the download | host the file yourself, or fetch it first |
| `TIMEOUT` | run exceeded its timeout | shard the media, or pick a faster preset |
| `OOM_LIMIT` | not enough memory | run with 16 GB |
| `INTERNAL` | unexpected failure | retry; report the run ID |

### Performance

Measured on a 16 GB Apify run (≈4 vCPU), CPU only:

| Media length | `fast` | `balanced` | `accurate` |
| --- | --- | --- | --- |
| 5 min | ~1 min | ~2 min | ~4 min |
| 30 min | ~5 min | ~10 min | ~20 min |
| 60 min | shard it | shard it | shard it |

For anything over ~30 minutes, extract 16 kHz mono audio first (see the
**Video & Audio Toolkit** actor) and transcribe in parallel windows.

### FAQ

**Does it need a GPU or an API key?** No. It runs whisper.cpp on CPU inside the actor.

**Which languages are supported?** All Whisper languages (90+). Leave
`language` empty to auto-detect, or set it explicitly for better accuracy.

**Can I get SRT/VTT?** The output gives word and segment timings in
milliseconds; building SRT/VTT from that is a few lines of code, and lets you
choose your own caption length.

**Can I transcribe a YouTube link directly?** Not here — resolve/download the
media first with the **Video Downloader** or **Video & Audio Toolkit** actor,
then pass the resulting URL as `source`.

**Are speaker names included?** No. Pair it with the **Speaker Diarization**
actor, which can label this transcript with who spoke each word.

# Actor input Schema

## `op` (type: `string`):

Transcribe audio/video, or return supported capabilities.

## `source` (type: `string`):

https:// URL or kv:<storeId>/<key>. Any audio or video file — it is converted to 16 kHz mono WAV automatically.

## `options` (type: `object`):

{ preset: fast|adaptive|balanced|accurate, language: 'vi', model, threads, timeout\_sec (0 = no internal limit), segment\_gap\_ms, keep\_raw\_json, start\_sec, duration\_sec, overlap\_sec (shard window — transcribe only \[start\_sec, start\_sec+duration\_sec+overlap\_sec]; result timestamps are offset back to the original timeline and words past duration\_sec are flagged \_overlap) }

## `output` (type: `object`):

{ signed\_upload\_url } to PUT transcript.json straight into your own storage.

## `cleanup` (type: `string`):

Cleanup policy

## `callback` (type: `object`):

{ url, secret\_header: { name, value } }

## Actor input object example

```json
{
  "op": "capabilities",
  "options": {
    "preset": "balanced",
    "language": "vi"
  },
  "cleanup": "on_success"
}
```

# Actor output Schema

## `results` (type: `string`):

Full run result JSON: status, artifacts \[{name, kv\_key, url, bytes}], meta, timings, errors.

## `resultRecord` (type: `string`):

The same result JSON stored as the RESULT record of the default key-value store.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "op": "capabilities",
    "options": {
        "preset": "balanced",
        "language": "vi"
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("andrew_babo/speech-to-text-whisper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "op": "capabilities",
    "options": {
        "preset": "balanced",
        "language": "vi",
    },
}

# Run the Actor and wait for it to finish
run = client.actor("andrew_babo/speech-to-text-whisper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "op": "capabilities",
  "options": {
    "preset": "balanced",
    "language": "vi"
  }
}' |
apify call andrew_babo/speech-to-text-whisper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,andrew_babo/speech-to-text-whisper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/R0oNMKNuyiaoEeZWf/builds/ta15Gw1wiCKkbOKn5/openapi.json