NHS Jobs Search Scraper

This Apify Actor extracts complete, publicly visible job adverts from NHS Jobs. It uses bounded direct HTTPS requests and Cheerio against the official search-results and candidate/jobadvert pages. No login, proxy, browser fingerprint spoofing, CAPTCHA solver, cloud call, or publish action is used.

Supported modes

• search: one query and optional location (the default).
• searchMultiple: several queries, deduplicated by NHS job ID while retaining matchedQueries.
• single: one official detail URL in jobUrl.
• multiple: several official detail URLs in jobUrls.
• startUrls: official NHS Jobs search-results and/or detail URLs in startUrls.

Every search candidate is verified through its official detail page before it is written. Search and detail work is bounded by maxItems, maxCandidates, maxPages, maxDetailRequests, and maxRequests.

Input examples

Search:

{
  "mode": "search",
  "query": "nurse",
  "location": "London",
  "maxItems": 3,
  "maxPages": 2
}

Single detail:

{
  "mode": "single",
  "jobUrl": "https://www.jobs.nhs.uk/candidate/jobadvert/A5412-26-0001"
}

The repository also contains INPUT-multiple.json, INPUT-search-multiple.json, INPUT-start-urls.json, and INPUT-negative.json for bounded local checks.

Output

Job records contain the stable NHS job ID, canonical/detail/source URLs, title, employer, public employer website and structured address, job location parts, summary and duties, employer information, detailed description text and HTML, parsed description sections, essential/desirable person-specification criteria, Disclosure and Barring Service/visa sponsorship/UK registration notices, pay scheme/band/salary, contract and working pattern, posted/closing/interview dates, supporting-document links and visible download labels, source-backed application URL, search attribution, request receipts, and data-quality metadata.

Direct personal contact fields are intentionally omitted. Placeholder criteria such as “Please see Job Description”, blank strings, nulls, empty arrays, and empty objects are omitted rather than turned into fabricated values. Diagnostics are written to key-value artifacts and never emitted as dataset rows.

examples/public-job-record.json is a complete source-backed example based on a local NHS Jobs run. The dataset validator checks this example and the selected run against the output schema, including more than 20 populated source fact groups per job.

RUN_SUMMARY, OUTPUT_SUMMARY, and RUN_HEALTH keep the Apify process status (SUCCEEDED or FAILED) separate from resultStatus (COMPLETE, LIMITED, SKIPPED, NO_DATA, or FAILED). limitedResults is true only when usable rows are preserved with a non-block issue. Confirmed source barriers stop requests, discard buffered rows, and produce SUCCEEDED/SKIPPED; clean empty crawls produce SUCCEEDED/NO_DATA; fatal non-block failures before any row produce FAILED/FAILED through Actor.fail().

Local validation

Run from this directory. Before apify run, inspect storage and legacy apify_storage and preserve any existing data. Select a fresh relative Actor storage path for each local validation run:

npm ci --ignore-scripts --no-audit --no-fund
npm test
npm run check
apify validate-schema
$env:APIFY_LOCAL_STORAGE_DIR = 'storage-nhs-local-validation'
apify run --resurrect --input-file INPUT.json
npm run validate -- storage-nhs-local-validation

The run writes the dataset and RUN_SUMMARY, OUTPUT_SUMMARY, RUN_DIAGNOSTICS, RUN_METADATA, and RUN_HEALTH to the selected local storage path. Never pass --purge or reuse an existing dataset directory. No Apify Cloud push or run is part of this validation workflow.

If the public site returns HTTP 401, 403, 429, or 451, or a visible access-denial, CAPTCHA, or human-verification page, the Actor stops requests, discards buffered job rows, and records SUCCEEDED/SKIPPED plus the diagnostic and receipts. A generic fatal failure before any complete record is stored produces FAILED/FAILED; a later non-block failure preserves stored rows as SUCCEEDED/LIMITED.

Runtime and output contract

The Actor supports bounded local and Apify cloud runs. The default dataset contains only complete job records; request failures and other diagnostics are retained in RUN_DIAGNOSTICS in the default key-value store. maxRequests caps total HTTP attempts, and deadlineSeconds caps wall-clock runtime. Raw input is not persisted in run metadata.