# CremyX ASR (whisper.cpp) (`andrew_babo/cremyx-asr`) Actor

Word-level speech-to-text with whisper.cpp. Accepts any audio or video (https URL or kv ref) and returns a word-level transcript JSON ready for clipping and subtitle pipelines.

- **URL**: https://apify.com/andrew\_babo/cremyx-asr.md
- **Developed by:** [Andrew Babo](https://apify.com/andrew_babo) (community)
- **Categories:** AI, Videos
- **Stats:** 23 total users, 19 monthly users, 53.5% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## cremyx-asr

whisper.cpp + model ggml nướng sẵn trong image. Thay cho việc bắt user tải
Whisper và model vài trăm MB về máy.

- Actor ID: `R0oNMKNuyiaoEeZWf` (`cremyx-asr`)
- Base image: `apify/actor-node:22` (Alpine) — whisper.cpp build **static**
- Build hiện tại: `0.1.4`, model mặc định `large-v3-turbo-q5_0` (~570 MB)

### Input

```jsonc
{
  "source": "kv:<storeId>/audio.wav",   // hoặc https://…  (mp4/m4a/mp3/wav đều được)
  "options": {
    "preset": "balanced",               // fast | adaptive | balanced | accurate
    "language": "vi",                   // bỏ trống = tự nhận diện
    "model": null,                      // tên file .bin khác trong image (nếu build nhiều model)
    "threads": null,                    // mặc định = adaptive theo số CPU
    "segment_gap_ms": 700,              // khoảng lặng để tách segment caption
    "keep_raw_json": false,             // giữ luôn JSON gốc của whisper.cpp
    "timeout_sec": null
  },
  "output": { "signed_upload_url": null },
  "cleanup": "on_success",
  "callback": null
}
```

Actor luôn tự convert nguồn về WAV 16 kHz mono trước khi chạy — đưa mp4 thẳng
vào cũng được, nhưng nếu đã có `audio16k` từ `cremyx-media-tools` thì nhanh hơn.

### Preset

| Preset | Dùng khi | Ghi chú |
|---|---|---|
| `fast` | duyệt nhanh, nội dung rõ tiếng | beam nhỏ |
| `adaptive` | mặc định an toàn cho video dài | tự cân theo CPU |
| `balanced` | **mặc định** — chất lượng/tốc độ tốt nhất | |
| `accurate` | lời thoại khó, nhiều tiếng ồn | chậm hơn đáng kể, bật `--dtw` nếu model hỗ trợ |

### Output

```jsonc
{
  "status": "success",
  "op": "transcribe",
  "artifacts": [{ "name": "transcript", "kv_key": "transcript.json", "url": "…" }],
  "meta": {
    "engine": "whisper.cpp", "model": "ggml-large-v3-turbo-q5_0.bin", "preset": "balanced",
    "language": "vi", "duration_sec": 5.7, "word_count": 18, "segment_count": 3,
    "words": [{ "id": 0, "text": "Xin", "startMs": 120, "endMs": 310, "confidence": 0.93 }],
    "segments": [{ "id": 0, "text": "Xin chào các bạn", "startMs": 120, "endMs": 1980 }]
  },
  "timings": { "decode_ms": 400, "asr_ms": 79000, "total_ms": 80100 },
  "errors": null
}
```

`words` dùng **đúng schema từ** của `engine/asr.mjs` bên SuperClipping
(`{ id, text, startMs, endMs, confidence }`) — bộ chuẩn hoá `normalizeWhisperJson`
được port nguyên văn, nên thay ASR local bằng actor này không phải sửa gì phía sau.

Dòng dataset được lược bớt `words`/`segments` cho nhẹ; bản đầy đủ nằm trong
record `RESULT` và artifact `transcript.json`.

### Hiệu năng & chi phí

- CPU 8 GB: ~79 s cho 5.7 s audio với `large-v3-turbo-q5_0` (phần lớn là nạp model).
  Video càng dài thì tỉ lệ càng tốt vì chi phí nạp model chỉ trả một lần.
- Video dài → **chạy song song nhiều run** (mỗi run một đoạn từ `op=cut`)
  thay vì tăng RAM một run.
- Muốn nhẹ và nhanh hơn: build lại image với model nhỏ hơn
  `apify push --build-arg WHISPER_MODEL=medium` (hoặc `small`, `base`).

### Ví dụ nối chuỗi

```js
const wav = await tools.mediaTools({ op: 'audio16k', source: 'https://…/video.mp4' });
const asr = await tools.asr({ source: wav.kv('audio'), options: { preset: 'balanced', language: 'vi' } });
console.log(asr.meta.words.length, 'từ');
```

### Shard window (v2) — video dài

Với nguồn dài (>3 phút), client chia shard qua `plan_shards` của cremyx-media-tools rồi chạy song song nhiều run, mỗi run:

```json
{ "source": "https://.../video.mp4",
  "options": { "start_sec": 120, "duration_sec": 120, "overlap_sec": 1, "language": "vi" } }
```

- Timestamp trong kết quả **đã được cộng offset về timeline gốc** (`meta.shard.offset_applied: true`).
- Từ nằm trong đuôi overlap có `_overlap: true` — client bỏ khi ghép shard.
- `timeout_sec: 0` tắt giới hạn nội bộ; mặc định = max(20 phút, 12× độ dài audio).
- Speaker không đổi giữa các shard; nếu cần speaker nhất quán toàn video thì dùng cremyx-diarize trên full file.

Memory mặc định run: 12 GB.

# Actor input Schema

## `op` (type: `string`):

Transcribe audio/video, or return supported capabilities.

## `source` (type: `string`):

https:// URL or kv:<storeId>/<key>. Any audio or video file — it is converted to 16 kHz mono WAV automatically.

## `options` (type: `object`):

{ preset: fast|adaptive|balanced|accurate, language: 'vi', model, threads, timeout\_sec (0 = no internal limit), segment\_gap\_ms, keep\_raw\_json, start\_sec, duration\_sec, overlap\_sec (shard window — transcribe only \[start\_sec, start\_sec+duration\_sec+overlap\_sec]; result timestamps are offset back to the original timeline and words past duration\_sec are flagged \_overlap) }

## `output` (type: `object`):

{ signed\_upload\_url } to PUT transcript.json straight into your own storage.

## `cleanup` (type: `string`):

Cleanup policy

## `callback` (type: `object`):

{ url, secret\_header: { name, value } }

## Actor input object example

```json
{
  "op": "transcribe",
  "options": {
    "preset": "balanced",
    "language": "vi"
  },
  "cleanup": "on_success"
}
```

# Actor output Schema

## `results` (type: `string`):

Full run result JSON: status, artifacts \[{name, kv\_key, url, bytes}], meta, timings, errors.

## `resultRecord` (type: `string`):

The same result JSON stored as the RESULT record of the default key-value store.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "options": {
        "preset": "balanced",
        "language": "vi"
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("andrew_babo/cremyx-asr").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "options": {
        "preset": "balanced",
        "language": "vi",
    } }

# Run the Actor and wait for it to finish
run = client.actor("andrew_babo/cremyx-asr").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "options": {
    "preset": "balanced",
    "language": "vi"
  }
}' |
apify call andrew_babo/cremyx-asr --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,andrew_babo/cremyx-asr"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/R0oNMKNuyiaoEeZWf/builds/nTM6ZBD9DYrSBXVlh/openapi.json