Apify Actors

AIROtec's Apify crawlers live in one pnpm and Nx workspace. Each Actor remains independently deployable while shared monitoring and release safety stay consistent.

Applications

App	Actor	Purpose
apps/wolt	wolt-restaurant-menu-scraper	Wolt restaurants and menus
apps/thehub	the-hub-jobs-scraper	TheHub job listings
apps/monday	monday-marketplace-scraper	Monday Marketplace apps

Booking is intentionally not part of this workspace.

Shared packages

• packages/actor-observability provides privacy-safe Sentry initialization, traces, metrics, outcome capture, and bounded shutdown.
• packages/actor-runtime owns expected-input outcomes, item limits, run summaries, terminal diagnostics, and the dataset-write/charge boundary.
• packages/actor-testkit validates Actor schemas, all dataset pages, and actual run statistics.
• packages/http-client provides bounded JSON requests and HTTP status errors; callers own retries.
• packages/apify-deployment provides Actor identity preflight, exact-tag candidate lookup, safe promotion, rollback, cleanup, and affected-app selection.
• tools/domain-sync deterministically updates TheHub position types and Monday categories without an AI service or personal access token.

Monday and TheHub clients stay inside their Actor apps. Domain-sync imports their ./client.js exports; Nx builds these dependencies before tooling runs. There are no separate site-client packages.

Run outcomes and limits

Every nonfatal run writes OUTPUT in its default key-value store with status, itemCount, failedRequests, and warnings. Status is completed, no_matches, input_rejected, or partial. Rejected input also includes a reason, actionable message, and optional details. Invalid input succeeds on Apify with that status message; it does not send a Sentry exception. Real infrastructure failures still fail the run.

Terminal request diagnostics are stored under stable failure-<hash> keys, not mixed into business datasets. Exhausted optional/detail requests and missing enrichment produce a partial summary. Fatal discovery failures still fail the run. Counters and warnings persist across migration through Apify state storage; this is not a transactional exactly-once output guarantee.

Limits count distinct items selected for scraping, not HTTP attempts or only successful outputs. 0 means unlimited for local and paid runs; hosted free-tier runs are capped at 50. MAX_PAGES_TEST uses the same validation and cap. Wolt retains its separate listing/detail charge events, after a successful dataset write.

Validate a deployed candidate with pnpm --dir apps/<actor> test:e2e --run <run-id>. Run validation reads actual duration, memory and persisted Crawlee request statistics; missing statistics fail validation. Passing a dataset ID instead checks data quality only and explicitly reports that performance was not checked.

Development

Use Node from .nvmrc and the Corepack-managed pnpm version declared in package.json.

corepack enable
pnpm install --frozen-lockfile
pnpm build
pnpm lint
pnpm format:check
pnpm test:unit
pnpm test:workspace

Live upstream checks are separate:

pnpm exec nx run-many -t test:integration --projects=wolt,thehub,monday

Run one Actor from its app directory, for example pnpm --dir apps/wolt start:dev.

Deployment

Pull requests run affected Nx checks. A successful merge to main selects affected Actors, verifies each configured opaque Actor ID and name, builds every candidate with one immutable tag, and runs app-specific E2E validation. Promotion starts only after every selected Actor validates; it is sequential and compensates definite partial failures by restoring captured latest builds.

Repository Actions require APIFY_TOKEN, THEHUB_ACTOR_ID, and MONDAY_ACTOR_ID. WOLT_ACTOR_ID is optional while the existing ID y0NfA98a3bpJBTodv remains current. Each Actor receives SENTRY_DSN through its Apify secret environment; no DSN is stored in this repository.

Actor names, ID-variable mappings, and E2E inputs live in packages/apify-deployment/src/actors.ts. Both candidate validation and cleanup resolve that registry; identity preflight and exact-tag promotion safeguards remain mandatory.

The one-time repository rename, deploy-all, verification, and source-repository archival are governed by docs/migrations/apify-actors-cutover.md.