test avatar

test

Pricing

Pay per usage

Go to Apify Store
test

test

Test

Pricing

Pay per usage

Rating

0.0

(0)

Developer

Big Soup

Big Soup

Maintained by Community

Actor stats

0

Bookmarked

1

Total users

0

Monthly active users

13 hours ago

Last modified

Categories

Share

actor: input-orchestrator

Single entry point for on-demand scrape batches. Post-pivot 4-case dispatch matrix (capacity × webhook):

payload vs capno webhookwebhook
fitssync (poll, return results)async (exit, deliverer fires later)
overflowFAIL no_capacityaccept (overflow queued for dispatcher), exit
  • Input: { target_app, payload[], webhook?, secret?, metadata? } — see .actor/input_schema.json
  • Output: Apify dataset entry — sync { batch_id, status, total, completed, failed, results[] } or async { batch_id, status: 'accepted', today, deferred }
  • DB writes: INSERT batches (encrypts webhook_secret_enc if secret provided) + RPC claim_scrape_slots (atomic + advisory locked)
  • Spawn fan-out: today-zone claims → 1 scrape-data actor per worker (with up to 5 jobs in jobs[])
  • Defer model: the deferred zone stores rows with worker_id IS NULL and queued_at=now; the dispatcher Edge Fn picks them up FIFO later (reactive on worker availability + hourly cron fallback)
  • Webhook off-ramp: PG trigger on batches.status='webhook_pending'/apify-webhook event batch.completed → spawns webhook-deliverer
  • Validation: payload 1..1000 items × 1..255 chars; webhook HTTPS + SSRF reject; secret only valid when webhook set; metadata max 2048 chars
  • Sync timeout: 15 min hard cap → batch flagged partial_timeout, partial results returned

See /Users/romanjanek/sm/ztest/memory.md for full system context.