# Video Render Engine: Timeline JSON to MP4 (`andrew_babo/video-render-engine`) Actor

Render a JSON edit timeline into an MP4: layered clips, crop and pan keyframes, captions, transitions and audio mixing, rendered headlessly and joined with ffmpeg. Scales across machines.

- **URL**: https://apify.com/andrew\_babo/video-render-engine.md
- **Developed by:** [Andrew Babo](https://apify.com/andrew_babo) (community)
- **Categories:** Videos, Developer tools
- **Stats:** 792 total users, 418 monthly users, 97.8% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Video Render Engine — Timeline JSON to MP4

Send a JSON edit timeline, get back a finished **MP4**. Layered clips, crop and
pan keyframes, captions, transitions and audio mixing are rendered frame by
frame in a headless browser, then encoded and joined with ffmpeg — so the output
matches what a browser preview shows, pixel for pixel.

**Use it for:** automated short-form video, subtitle burn-in, vertical 9:16
reframing at scale, templated social clips, batch rendering from a CMS or an
AI pipeline.

- Declarative JSON in, MP4 out — no timeline software, no GPU, no local ffmpeg
- Crop / pan / zoom **keyframes** with linear or hold interpolation
- Burned-in **captions** with word timing
- Audio mixing: voice gain, source gain, background music, loudness normalisation
- Long renders are split into shards and rendered in parallel, then joined losslessly

### Quick start

```json
{
  "mode": "editplan",
  "source_url": "https://example.com/master.mp4",
  "edit_plan": {
    "canvas": { "width": 1080, "height": 1920, "fps": 30 },
    "tracks": [
      {
        "type": "media",
        "clips": [
          {
            "source": "https://example.com/master.mp4",
            "start_ms": 0,
            "end_ms": 61400,
            "layout": "single-center",
            "crop": {
              "keyframes": [
                { "t_ms": 0,     "rect": { "x": 0.31, "y": 0, "w": 0.316, "h": 1 }, "interp": "linear" },
                { "t_ms": 61400, "rect": { "x": 0.36, "y": 0, "w": 0.316, "h": 1 }, "interp": "linear" }
              ]
            }
          }
        ]
      }
    ]
  }
}
```

Feature-detect the build (free, a few seconds, needs no plan):

```json
{ "mode": "capabilities" }
```

### Input

| Field | Notes |
| --- | --- |
| `mode` | `editplan` (default) or `capabilities` |
| `edit_plan` | the timeline document, inline |
| `edit_plan_url` | URL of the timeline JSON, when it is too big to inline |
| `source_url` | shorthand when the plan has exactly one source |
| `source_map` | `{ "<sourceId>": "https://…" }` for multi-source plans |
| `video` | `{ width, height, fps, crf, preset }` — defaults come from the plan canvas |
| `audio` | `{ voice_gain_db, source_gain, bgm_url, bgm_gain_db, loudnorm }` |
| `options` | `{ workers, threads, profile, preset, crf, min_shard_sec, captions, maxAssetBytes }` |
| `output` | `{ signed_upload_url, fallback_kv_key }` |
| `callback` | webhook called with the result JSON when the run finishes |

#### Timeline basics

- `canvas` — output size and frame rate; everything else is expressed relative to it.
- `tracks[]` — layered top to bottom; media, caption and overlay tracks.
- `clips[]` — `source`, `start_ms`, `end_ms`, `layout`, and optional
  `crop`, `transform`, `opacity`, `transition`, `filters`.
- `crop.keyframes[]` — `rect` in **normalised 0–1 coordinates**, so the same plan
  renders at any resolution. This is exactly the format the
  **Face Detection & Auto Reframe** actor produces, so auto-reframe output can be
  pasted in directly.

### Output

```json
{
  "status": "success",
  "artifacts": [
    { "name": "video", "kv_key": "output.mp4", "url": "https://api.apify.com/v2/key-value-stores/.../output.mp4", "bytes": 18442310 }
  ],
  "meta": {
    "width": 1080, "height": 1920, "fps": 30,
    "duration_sec": 61.4,
    "frame_count": 1842,
    "shards": 4,
    "render_sec": 96.3
  }
}
```

Set `output.signed_upload_url` to have the MP4 PUT straight into your own
storage bucket instead of staying on Apify.

### Error handling

```json
{ "ok": false, "reason": "BAD_INPUT", "message": "..." }
```

| `reason` | Meaning | What to do |
| --- | --- | --- |
| `BAD_INPUT` | invalid plan, unknown source id, bad keyframes | validate the plan; check every `source` resolves |
| `UPSTREAM_BLOCKED` | a source URL could not be fetched | host the media somewhere publicly reachable |
| `TIMEOUT` | render exceeded the run timeout | raise `timeoutSecs`, or lower resolution / fps |
| `OOM_LIMIT` | not enough memory | run with 16 GB |
| `INTERNAL` | unexpected failure | retry; report the run ID |

### Performance

16 GB run (≈4 vCPU), 1080×1920 at 30 fps:

| Output length | Typical time |
| --- | --- |
| 15 s | ~25–40 s |
| 60 s | ~1.5–3 min |
| 5 min | ~8–15 min |

Rendering is split into shards across the available vCPUs (`options.workers`)
and the shards are concatenated with a stream copy, so there is no second
encode. 16 GB and a run timeout of 2 hours are the recommended settings.

### Pipeline example

1. **Video Downloader** — fetch the source MP4 from a page URL.
2. **Video & Audio Toolkit** — build a 480p proxy and a 16 kHz mono audio track.
3. **Speech to Text (Whisper)** — get word timestamps for the captions.
4. **Face Detection & Auto Reframe** — get 9:16 crop keyframes that follow the speaker.
5. **Video Render Engine** — assemble the plan and render the final vertical MP4.

### FAQ

**Do I need ffmpeg or an editor installed?** No. Everything happens in the actor.

**Why a headless browser?** So the rendered frames use the same drawing rules as
a browser-based preview — one implementation instead of two that drift apart.

**Can I burn in subtitles?** Yes — add a caption track with word timings, or
enable `options.captions`.

**Can I add background music?** Yes — `audio.bgm_url` plus `bgm_gain_db`, with
optional `loudnorm` for consistent loudness.

**Is the output web-ready?** Yes — H.264/AAC MP4 with faststart.

#### Building from source

`src/generated/` and `src/vendor/` are generated; do not edit them by hand.
Regenerate from the repository root with:

```bash
bun scripts/build-actor-frames.mjs
```

# Actor input Schema

## `mode` (type: `string`):

editplan (default) or capabilities.

## `edit_plan` (type: `object`):

The EditPlan document to render.

## `edit_plan_url` (type: `string`):

URL of the EditPlan JSON, used when edit\_plan is not sent inline.

## `source_url` (type: `string`):

Shorthand for plans with exactly one source.

## `source_map` (type: `object`):

sourceId -> media URL.

## `video` (type: `object`):

width, height, fps, crf, preset. Defaults come from the plan canvas.

## `audio` (type: `object`):

voice\_gain\_db, source\_gain, bgm\_url, bgm\_gain\_db, loudnorm.

## `output` (type: `object`):

signed\_upload\_url or fallback\_kv\_key.

## `options` (type: `object`):

workers, threads, profile, preset, crf, min\_shard\_sec, captions, maxAssetBytes.

## `callback` (type: `object`):

Optional webhook called with the result.

## Actor input object example

```json
{
  "mode": "capabilities"
}
```

# Actor output Schema

## `results` (type: `string`):

Full run result JSON: status, engine fingerprint, plan meta, timings, errors.

## `resultRecord` (type: `string`):

The same result JSON stored as the RENDER\_RESULT record of the default key-value store.

## `renderedVideo` (type: `string`):

The rendered MP4 record in the default key-value store (when the run keeps the output in-store).

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "mode": "capabilities"
};

// Run the Actor and wait for it to finish
const run = await client.actor("andrew_babo/video-render-engine").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "mode": "capabilities" }

# Run the Actor and wait for it to finish
run = client.actor("andrew_babo/video-render-engine").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "mode": "capabilities"
}' |
apify call andrew_babo/video-render-engine --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,andrew_babo/video-render-engine"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/8HM6wVpUPjhmgq75q/builds/3KYDrgr0fOMsuD91s/openapi.json