CremyX Render Engine (Apify Actor)

FFmpeg-based render engine that runs on Apify instead of an edge function. Phase 0 scope: build the actor, run POCs. Nothing in the main app calls it yet.

Modes

mode	input per scene	what it does
concat	video_url + optional voice_url	normalises each clip to the canvas, trims/frame-pads it to the voice length, ducks the clip's own audio under the voice, then joins everything with the concat demuxer
kenburns	image_url + optional voice_url	linear zoom on a supersampled canvas scaled straight to output size (no intermediate rescale, so no jitter)
editplan	a whole *.editplan.json	renders a Juhe/Velorn EditPlan document: regions, per-clip crop, keyframed pan, split layouts, audio tracks

concat / kenburns then lay the optional BGM under the whole timeline with sidechain ducking, fades and loudnorm I=-19 TP=-0.6, and verify the output with ffprobe before uploading.

editplan mode (Juhe / Velorn)

{
  "mode": "editplan",
  "edit_plan_url": "https://…/clip_01.editplan.json",
  "source_map": { "src_main": "https://…/source.mp4" },
  "output": { "fallback_kv_key": "clip_01.mp4" }
}

edit_plan (inline object) works too. source_map is required whenever the plan points at a local path (file:///D:/…) — the actor only reads http(s). Keys may be the source id or its original uri.

Supported today:

area	supported	not yet
layout	single-center, blur-pad, split-vertical/horizontal, any layout.regions rectangle	free-form rotation
transform	sourceCropRect, keyframed translate and scale (real moving zoom) with hold / linear easing	rotationDeg != 0, non-linear easing curves (approximated as linear, reported)
frameFit	cover, plus the blurred/dimmed backdrop for pad modes	—
audio	audio clips with gainDb, placed on the timeline, optional BGM + loudnorm	per-clip fades declared in the plan
captions	plain caption clips burned with a built-in ASS style	the Juhe title pack (canvas/JS renderer — Phase 2, needs Chromium in the image)

Anything in capabilitiesRequired the actor cannot honour aborts the run with the exact list; pass ignore_capabilities: true to render anyway.

The result record gains a plan block: segment/clip counts, planned duration and the warning list.

Input

{
  "mode": "concat",
  "scenes": [
    { "id": 1, "video_url": "https://…/clip1.mp4", "voice_url": "https://…/voice1.mp3" },
    { "id": 2, "video_url": "https://…/clip2.mp4", "voice_url": "https://…/voice2.mp3" }
  ],
  "video": { "width": 1920, "height": 1080, "fps": 30, "crf": 20, "preset": "fast" },
  "options": {
    "profile": "balanced",
    "workers": 4,
    "threads": 2,
    "shard_sec": 0
  },
  "audio": { "voice_gain_db": 4, "source_gain": 0.1, "bgm_url": null, "bgm_gain_db": -22, "loudnorm": true },
  "output": { "signed_upload_url": null, "fallback_kv_key": "output.mp4" },
  "callback": { "url": null, "secret_header": null }
}

options.profile is optional: balanced (default for editplan, recommended), quality (master), fast (quick turnarounds), or draft (internal preview). Explicit video.preset/video.crf or options.preset/options.crf override the profile. workers and threads override the pool only when you have measured the workload. shard_sec: 0 disables straggler splitting; otherwise the engine splits only segments longer than the total timeline divided by the worker count.

Per-scene extras: duration_sec (overrides the voice-derived length) and, for kenburns, zoom_to.

Output

• MP4 is PUT to output.signed_upload_url when present, otherwise stored in the run's key-value store under output.fallback_kv_key.
• A summary record is pushed to the dataset and stored as RENDER_RESULT: { status, mode, duration_sec, size_bytes, width, height, has_audio, output_url, stored_in, timings, errors }.
• callback.url (optional) receives the same JSON via POST.

Build & run

cd apify-render-engine
npx apify-cli login --token "$APIFY_BUILD_TOKEN"
npx apify-cli push                 # creates the actor + a build
npx apify-cli call cremyx-render-engine --input-file poc/poc1-smoke.json

The token lives in the project secret APIFY_BUILD_TOKEN — never commit it.

POC checklist

1. poc/poc1-smoke.json — 2 clips + 1 voice, no BGM, KV output.
2. Real storyboard project (25 clips + voice + BGM) — compare duration, loudness (~-19 LUFS), peak, file size and transition smoothness against the offline render.
3. poc/poc3-kenburns.json — stills + voice, check there is no zoom jitter.

Record run time and Compute Units for each POC before moving to Phase 1.

Phase 0 results (2026-08-30)

Actor: Andrew_babo/cremyx-render-engine — id 8HM6wVpUPjhmgq75q, build 0.1.3 (tag latest).

POC	input	memory	run time	Compute Units	output
1 smoke (concat + BGM)	2 clips, 2 voices, 720p	4 GB	9.0 s	0.0099 CU	9.68 s / 3.0 MB
2 real storyboard	25 clips + 25 voices, cremyx-gioi-thieu-kraft, 1080p crf 20	8 GB	149.6 s	0.332 CU	56.40 s / 33.2 MB, integrated loudness -18.6 LUFS
3 ken burns	2 stills + voices, 1080p, 8K zoom canvas	8 GB	13.1 s	0.029 CU	9.68 s / 1.8 MB, no zoom jitter

Cost signal: a full 25-scene 1080p project ≈ 0.33 CU (~2.5 minutes wall clock at 8 GB).

Rotate APIFY_BUILD_TOKEN — it was pasted in chat during Phase 0.

Phase 1a results (2026-08-30, local verification)

Both uploaded EditPlans were rendered locally against a synthetic 1280x720/90 s stand-in source (the plans' real source is a local Windows file):

plan	canvas	planned	rendered	segments / clips	notes
clip_01	1080x1920@30	74.193 s	74.199 s	5 / 5 video, 1 audio	blur-pad shots, 216 keyframes of pan honoured
clip_02	1080x1920@30	81.729 s	81.699 s	5 / 7 video, 1 audio	split-vertical shots render two regions per segment

Duration error < 0.05 s in both cases; a sampled frame of clip_02 at 60 s shows the two split regions filling the top and bottom halves correctly.

ffmpeg 8 removed crop=…:eval=frame; the actor probes the filter once and only adds the option on ffmpeg <= 7, so animated pans work on both.

EditPlan POC on Apify (build 0.1.5)

poc/poc4-editplan.json — 20 s slice of clip_02 against a public 1920x1080 sample source, 8 GB memory: SUCCEEDED, 20.00 s / 1080x1920 / 20.0 MB, 57.4 s render, 0.141 CU. The mismatched plan resolution was auto-corrected from the probed file and reported as a warning.

Dense editor keyframes (one every 0.2 s) are simplified to the points that actually change the curve before the ffmpeg expression is built — 161 keyframes became a 362-character expression. Without that, older ffmpeg builds refuse to configure the crop filter.

Build 0.1.6 (2026-08-30)

The video input no longer carries a schema default. In editplan mode the plan's canvas always wins unless the caller explicitly passes video — previously the injected 1920x1080 default silently overrode vertical plans.

Phase 2a — SuperClipping cloud render (2026-08-30)

Patch package superclipping-cloud-render/ (repo root, delivered as superclipping-cloud-render-v1.zip) adds a "Render trên cloud (Apify)" button to the SuperClipping Export screen: it uploads the source video to an Apify KV store, starts one editplan run per clip_XX.editplan.json (4 in parallel), then downloads the MP4s back into the project folder. Apify token is stored encrypted in a separate cloud-settings.json (safeStorage), never in the app's settings.json.

End-to-end POC (Node harness driving the real cloudRender.cjs IPC handlers, build 0.1.6): 10 s blur-pad plan, 1080x1920@30 output, SUCCEEDED in 12.8 s wall clock, 0.011 CU, MP4 downloaded and verified with ffprobe.

Captions, frame fit, animated zoom (v0.1.11)

• caption tracks render word-level burned-in ASS subtitles. Words accept startSec/endSec (or start/end, tStartSec/tEndSec). The active word is highlighted with style.activeColor; markers: *word* = accent colour, _word_ = minor colour, trailing | = line break.
• Style: fontSizeRatio, activeColor, accentColor, minorColor, uppercase, position.anchor + position.y (fraction of canvas height).
• Custom fonts: options.fonts: ["https://.../Font.ttf"]. The image also ships DejaVu + Noto, so Vietnamese diacritics work out of the box.
• frameFit: cover (default), contain, blur, color. contain/color no longer report a false capability warning.
• Animated scale/translate keyframes render as real zoom/pan (widest crop + zoompan with a PTS reset), so 1 → 1.05 no longer collapses to a static crop.
• Encoding: options.preset (default veryfast) and options.crf (default 20). ffmpeg threads follow the Apify memory quota, not host os.cpus().
• Results are written to both RENDER_RESULT and OUTPUT key-value keys.

Transform keyframes: accepted field names + diagnostics (v0.1.16)

A keyframe whose time field is not recognised used to land on t=0, so every keyframe collapsed onto one instant and the zoom looked frozen (or snapped to the last value). The parser now reads every spelling in the wild and reports what it found.

slot	accepted fields
time	tSec, t_sec, timeSec, atSec, offsetSec, startSec; ms variants tMs, timeMs, atMs, offsetMs, ms; ambiguous t, time, at, offset, position, pos, timestamp
zoom	scale, zoom, scaleFactor, zoomFactor; percent variants scalePercent, zoomPercent
pan	translate / translation / offset / pan / position {x,y}, or translateX/translateY, tx/ty, panX/panY, x/y
easing	easing, interpolation, interp, curve, mode — hold/step/constant hold, everything else interpolates linearly
rotation	rotationDeg, rotation, rotate, angleDeg (still rejected when non-zero)

Nested carriers are read too: { transform: { scale } }, { value: { scale } }, { props: { … } }.

Unit inference, all reported in the diagnostics:

• ambiguous time fields are read as milliseconds when the largest value is more than 3x the clip duration and sane after /1000, else as seconds;
• pan values are fractions of the source size, unless unit: "px"/"percent" is declared or every value exceeds ±2 (then they are pixels).

plan.transform_diagnostics[] (one row per clip per segment) carries time_field, time_unit, scale_field, translate_field, translate_unit, zoom_range, time_range, segment_offset_sec, animated_scale, collapsed, unsupported_easings, notes. plan.animated_zoom_clips and plan.collapsed_keyframe_clips are the roll-ups. A collapsed clip also raises a warning instead of silently rendering a static frame, and holds the widest framing rather than snapping to the last keyframe.

Two related fixes shipped with it:

• keyframe times are now shifted by -segment_offset (they were shifted the wrong way), so a clip that spans a segment boundary keeps one continuous zoom instead of restarting at every cut;
• a full-canvas region is overlaid at 0:0 (it was rounded up to 2:2).

Speed: parallel captions, stream-copy concat, straggler sharding (v0.1.17 – v0.1.21)

P0-6 asked for faster editplan renders. Three changes did the work:

1. Captions burn inside the parallel segment pass. They used to run as a second full-timeline libass encode after the concat, i.e. the whole video was encoded twice. Word captions now cost ~0 wall time.
2. Stream-copy concat. Segments are already encoded with identical parameters, so the join is -c copy instead of a re-encode.
3. Straggler-only sharding. The ideal wall time is total / workers; only segments longer than that are split (shardSegments). Sharding every segment was slower — each ffmpeg start costs ~1.5 s on Apify (binary load + fontconfig).

Sources download in parallel, and the worker pool matches the vCPU count the memory tier actually grants (16 GB → 4 vCPU → 4 workers × 2 threads).

Measured on Apify, 16 GB / 4 vCPU

60 s output, 1080x1920 @ 30 fps, moving zoom on every clip + word captions:

profile	preset / crf	wall time	vs realtime	size
quality	medium / 19	120 s	2.00x	73.5 MB
balanced (default)	veryfast / 20	51 s	0.85x	58.5 MB
fast	superfast / 21	43 s	0.69x	58.5 MB
draft	ultrafast / 23	31 s	0.52x	97.8 MB

Baseline before this work: 108 s (1.80x realtime) for the same plan. fast is the sweet spot on this content — same file size as balanced, 20 % quicker. draft trades 1.7x file size for the fastest turnaround.

Why not ≤0.5x on balanced

Profiling (local, same plan) puts >60 % of the time in x264 itself and the rest in zoompan + libass. The 16 GB tier is capped at 4 vCPU, so ≤0.5x at veryfast needs roughly 8 vCPU. Options, in order of preference: run fast (0.69x today), move to a 32 GB tier when Apify grants more cores, or accept draft for previews. Nothing in the pipeline is serialised any more — adding cores scales close to linearly because segments are independent.

Capabilities

{"mode":"capabilities"} now reports version: "0.4", the speed_profiles table, and the feature flags parallelCaptionBurn, streamCopyConcat, stragglerSharding, speedProfiles, workerOverrides, alongside the existing caption/transform flags.