Jobs.ac.uk Jobs Search Scraper

This Actor performs bounded extraction of public Jobs.ac.uk academic job pages locally or on Apify Cloud. It uses ordinary HTTPS requests with a browser-compatible user agent and Cheerio parsing. It does not sign in, use fingerprinting or stealth plugins, submit applications, or fetch external application destinations.

Modes

• search discovers public Jobs.ac.uk vacancies through the official search page, then fetches each detail page.
• single fetches one URL from jobUrl, directUrl, or startUrl.
• multiple fetches URLs from jobUrls, directUrls, or startUrls.

Every emitted row is buffered until its detail page, canonical URL, job identity, description, and required fields are verified. Incomplete or mismatched jobs are skipped in search mode; transport, access, and source-integrity failures write structured diagnostics to the key-value store and emit no misleading row.

Output

Rows preserve the public academic vacancy data available on the page: title, institution, department, full text and HTML description, sectioned responsibilities/qualifications/benefits, locations, salary, hours, contract type, posting and closing dates, job reference, structured JobPosting data, description links, contact emails, explicit external application links, source listings, request receipts, field coverage, and dynamic execution-quality evidence.

applicationUrl is emitted only when Jobs.ac.uk publishes an explicit application action. The canonical Jobs.ac.uk vacancy URL is never presented as an application link.

Local run

npm ci
apify run --purge --input-file INPUT.json
npm run validate

The reproducible fixtures INPUT.json, INPUT-single.json, and INPUT-multiple.json exercise all three modes. The dataset is stored in storage/datasets/default; run state and diagnostics are stored under storage/key_value_stores/default.

On Apify Cloud, deploy with apify push and use a bounded 512 MB, 300-second smoke input. OUTPUT_SUMMARY, RUN_HEALTH, RUN_DIAGNOSTICS, RUN_SKIPS, and SOURCE_RECEIPTS provide dynamic run/build/storage provenance and safe response evidence.

The validator rejects nulls, blank strings, placeholders, empty arrays/objects, duplicate IDs/URLs, non-Jobs.ac.uk URLs, inconsistent URL identity, and unverifiable records. A zero-row run is accepted only when OUTPUT_SUMMARY records a structured failure diagnostic.

Bounded settings

maxItems is limited to 10, maxPages to 3, detail concurrency to 3, request timeout to 30 seconds, retries to 2, and optional delay to one second. The default fixtures complete well under four minutes. A target-site timeout or security response is reported honestly; proxy escalation is considered only when direct cloud evidence justifies it.