Speaker Diarization — Who Spoke When in Audio & Video

Find out how many speakers are in a recording and when each one talks. Runs pyannote segmentation plus speaker embeddings through sherpa-onnx on CPU — no GPU, no API keys, no local install.

Optionally pass an existing transcript and get the same words and segments back with a speaker label attached, ready for a two-column interview transcript.

Use it for: interview and podcast transcripts, meeting minutes, call-centre QA, subtitle attribution, dataset labelling, audio search.

Quick start

{
  "op": "diarize",
  "source": "https://example.com/interview.mp3"
}

Feature-detect the build (free, a few seconds, needs no source):

{ "op": "capabilities" }

Label an existing transcript:

{
  "op": "diarize",
  "source": "https://example.com/interview.mp3",
  "transcript": "https://api.apify.com/v2/key-value-stores/.../transcript.json",
  "options": { "num_speakers": 2 }
}

Input

Field	Type	Notes
op	diarize | capabilities	default diarize
source	string	https:// URL or kv:<storeId>/<key>. Any audio or video; converted to 16 kHz mono internally.
transcript	string	optional transcript JSON to label (word/segment timings in ms)
options.num_speakers	integer	set it when you know the exact count — most accurate
options.cluster_threshold	number	default 0.5; used when num_speakers is unset
options.min_duration_on / min_duration_off	number	minimum speech / silence length
options.threads	integer	defaults to the run's vCPU count
options.start_sec / duration_sec / overlap_sec	number	diarize one window only
output.signed_upload_url	string	PUT diarization.json into your own storage
cleanup	on_success | always | off	artifact retention
callback	object	{ url, secret_header: { name, value } } webhook

Output

{
  "status": "success",
  "op": "diarize",
  "artifacts": [{ "name": "diarization", "kv_key": "diarization.json", "url": "https://api.apify.com/v2/key-value-stores/.../diarization.json" }],
  "meta": {
    "duration_sec": 1820.5,
    "speaker_count": 2,
    "turns": [
      { "speaker": "SPEAKER_00", "start_ms": 0,     "end_ms": 8450 },
      { "speaker": "SPEAKER_01", "start_ms": 8600,  "end_ms": 15120 }
    ]
  }
}

With transcript set, the artifact also contains the input words and segments with a speaker field added to each one.

Speaker IDs are stable within one run only. If you shard a long file, match speakers across shards yourself (for example by overlapping a few seconds and comparing who is talking there).

Sharding a long recording

{
  "op": "diarize",
  "source": "https://example.com/long.wav",
  "options": { "start_sec": 1800, "duration_sec": 900, "overlap_sec": 5 }
}

Turns are offset back to the original timeline; items past duration_sec are flagged _overlap.

Error handling

{ "ok": false, "reason": "BAD_INPUT", "message": "..." }

reason	Meaning	What to do
BAD_INPUT	missing/unreadable source or transcript	check URLs are publicly reachable
UPSTREAM_BLOCKED	the host refused the download	host the file yourself first
TIMEOUT	run exceeded its timeout	shard the recording
OOM_LIMIT	not enough memory	run with 16 GB
INTERNAL	unexpected failure	retry; report the run ID

Performance

16 GB run (≈4 vCPU), CPU only:

Recording	Typical time
10 min	~1 min
60 min	~5–8 min
3 h	shard into 15–30 min windows

Extracting mono 16 kHz audio first (see the Video & Audio Toolkit actor) makes the download and conversion step much cheaper for video sources.

FAQ

Does it recognise who the people are? No — it separates speakers and labels them SPEAKER_00, SPEAKER_01… Mapping labels to real names is up to you.

How accurate is the speaker count? Very good for 2–4 clean speakers. Set num_speakers when you know it; heavy crosstalk and noisy phone audio are the hard cases.

Can it label an existing transcript? Yes — pass transcript and every word and segment comes back with a speaker field.

Video input? Supported; the audio track is extracted automatically.

Need visual speaker detection? The Face Detection & Auto Reframe actor detects the active speaker from mouth movement — the two signals complement each other.