# NHS Jobs Scraper — Job Listings & Career Data Extractor (`scrapepilot/nhs-uk-public-job-board-scraper`) Actor Scrape NHS job listings from jobs.nhs.uk. Get title, employer, salary, NHS band, location, contract type, closing date, job description, main duties, and structured person specification. Auto-pagination. Checkpoint resume. $2 per 1,000 jobs. - **URL**: https://apify.com/scrapepilot/nhs-uk-public-job-board-scraper.md - **Developed by:** [Scrape Pilot](https://apify.com/scrapepilot) (community) - **Categories:** Jobs, Automation, Developer tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing from $1.50 / 1,000 scraped results This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README --- ## 🏥 NHS Job Scraper – Extract Job Details, Employer, Salary, Band, Location & More **Scrape NHS job vacancies from `jobs.nhs.uk` – structured data including job title, employer, salary, band, location, contract type, working pattern, closing date, job summary, main duties, and person specification.** Pay only for successfully scraped jobs ($2 per 1,000). Automatic checkpoint/resume, residential proxy ready. Perfect for recruitment analytics, labour market research, and job board aggregation. --- ### 💡 What is the NHS Job Scraper? The **NHS Job Scraper** extracts **full job listing data** from the official NHS Jobs portal (`jobs.nhs.uk`). It starts from a search results URL (e.g., with keywords, location, or filters) and then: - Crawls paginated search result pages - Visits each job detail page - Parses all available structured and unstructured fields The actor extracts every piece of public information from a typical NHS job advert: - **Job title** - **Employer (Trust/Organisation)** - **Salary** (normalised, including band detection) - **Band** (e.g., `Band 5`, `Band 6`) - **Location** (town/city) - **Working pattern** (full‑time, part‑time, flexible) - **Contract type** (permanent, fixed‑term, secondment) - **Closing date** and **date posted** - **Reference number** - **Job summary** (short overview) - **Main duties** (bulleted list) - **Person specification** (structured: categories + essential/desirable criteria) The **NHS Job Scraper** is built for **pay‑per‑event (PPE)** – you are charged **only for successfully scraped job adverts** ($0.002 per job). Failed requests, blocked pages, or missing data cost nothing. The actor also includes **automatic checkpoint/resume** – if interrupted (spending limit, timeout, network error), it restarts from the last page without re‑scraping already processed jobs. --- ### 🚀 Key Features | Feature | Description | |-----------------------------|-------------| | **Start from any search URL** | Use NHS Jobs search filters (keyword, location, distance, etc.). | | **Paginated crawling** | Automatically follows `page=X` parameters up to `maxItems`. | | **Full job detail extraction** | Title, employer, salary, band, location, contract, working pattern, closing date, reference. | | **Long‑text fields** | Job summary, main duties, job description. | | **Person specification** | Structured output (category → list of essential/desirable criteria). | | **Normalised salary & band** | Converts salary text to a consistent format; detects NHS Bands automatically. | | **Resume & checkpoint** | Saves progress after each page. If the actor stops, it restarts where it left off. | | **Pay‑per‑success (PPE)** | Charged **only** for jobs that are successfully scraped and pushed to dataset. Failed attempts are free. | | **Spending limit protection** | Stops automatically when the user’s daily/monthly limit is reached. | | **Residential proxy ready** | Bypasses anti‑bot measures (recommended for large runs). | | **Clean JSON / CSV / Excel** | Export from Apify dataset. | --- ### 📥 Input Parameters The actor accepts a JSON object with the following fields: | Parameter | Type | Required | Default | Description | |----------------------|---------|----------|---------|-------------| | `startUrl` | string | **Yes** | – | A valid NHS Jobs search URL (e.g., `https://www.jobs.nhs.uk/candidate/search/results?keyword=nurse&page=1`). | | `maxItems` | integer | No | `2000` | Maximum number of jobs to scrape (actor stops when reached). | | `proxyConfiguration` | object | No | – | Apify proxy configuration. **Residential proxies strongly recommended** to avoid blocking. | #### Example Input ```json { "startUrl": "https://www.jobs.nhs.uk/candidate/search/results?keyword=physiotherapist&location=London&page=1", "maxItems": 500, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"] } } ```` *** ### 📤 Output Fields Each successful job advert returns an object with the following fields (some may be `null` or `[]`): | Field | Type | Description | |---------------------------|---------|-------------| | `url` | string | Direct link to the job advert. | | `id` | string | Job ID (extracted from URL). | | `title` | string | Job title. | | `employer` | string | NHS Trust or organisation name. | | `salary` | string | Normalised salary (e.g., `£28,407 - £34,581` or `Band 5`). | | `band` | string | NHS band (e.g., `Band 5`, `Band 6`). | | `location` | string | Primary location (town/city). | | `pay_scheme` | string | Pay scheme (e.g., `Agenda for Change`). | | `working_pattern` | string | Working pattern (e.g., `Full time`, `Part time`, `Flexible working`). | | `contract_type` | string | Contract type (e.g., `Permanent`, `Fixed term`, `Secondment`). | | `closing_date_text` | string | Closing date (as displayed). | | `date_posted_text` | string | Date posted (as displayed). | | `reference_number` | string | Job reference number (falls back to URL id). | | `job_summary` | string | Short job summary paragraph. | | `main_duties_of_the_job` | string | Main duties / responsibilities. | | `job_description` | string | Combined job summary + main duties (if separate description missing). | | `person_specification` | array | List of categories with essential/desirable items (structured). | #### Example Output ```json [ { "url": "https://www.jobs.nhs.uk/candidate/jobadvert/A1234-5678", "id": "A1234-5678", "title": "Senior Occupational Therapist", "employer": "Guy's and St Thomas' NHS Foundation Trust", "salary": "£40,057 - £45,839 per annum incl. HCA", "band": "Band 6", "location": "London SE1", "pay_scheme": "Agenda for Change", "working_pattern": "Full time - 37.5 hours per week", "contract_type": "Permanent", "closing_date_text": "15 June 2026", "date_posted_text": "22 May 2026", "reference_number": "A1234-5678", "job_summary": "We are looking for an experienced Occupational Therapist to join our dynamic team...", "main_duties_of_the_job": "• Manage a complex caseload • Provide clinical supervision • Participate in service development", "job_description": "We are looking for... • Manage a complex caseload...", "person_specification": [ { "category": "Qualifications", "essentials": ["Degree in Occupational Therapy", "HCPC registration"] }, { "category": "Experience", "essentials": ["Minimum 2 years post‑graduate experience", "Experience in acute medicine"], "desirables": ["MSc in relevant field"] } ] } ] ``` *** ### 💰 Pricing | Component | Price | |-------------------------------|-----------------------| | **Actor start (per run)** | **$0.40** | | **Per successful job** | **$0.002** | | **Per 1,000 successful jobs** | **$2.00** | - You are **charged only when a job advert is successfully scraped** (title exists, basic fields present). - Failed pages (blocked, no results, 404) cost nothing. - The actor start fee ($0.40) covers infrastructure for the run (even if no jobs are scraped). - Example: 500 successful jobs = $0.40 + (500 × $0.002) = **$1.40**. - Example: 2,500 successful jobs = $0.40 + (2,500 × $0.002) = **$5.40**. **Checkpoint & Resume:**\ The actor saves state after every page. If it stops due to spending limit, timeout, or network issue, restarting will continue from the last page **without re‑charging** for already processed jobs. *** ### 🛠 How to Use on Apify 1. **Create a task** with this actor. 2. **Get a search URL** from NHS Jobs – go to `jobs.nhs.uk`, enter your search criteria (keyword, location, distance, etc.), copy the URL from the results page (it should contain `page=1`). 3. **Paste the `startUrl`** into the input. 4. **Set `maxItems`** – how many jobs to scrape (max 2000 per run, but you can run multiple tasks). 5. **Enable residential proxies** – go to **Proxy Configuration** → **Selected proxies** → **Residential** (strongly recommended to avoid blocking). 6. **Run** – the actor will crawl pages, extract each job, push to dataset, and charge per successful job. 7. **Export** – download results as JSON, CSV, or Excel from the **Dataset** tab. #### Running via API ```bash curl -X POST "https://api.apify.com/v2/acts/your-username~nhs-job-scraper/runs" \ -H "Content-Type: application/json" \ -H "Authorization: Bearer YOUR_API_TOKEN" \ -d '{ "startUrl": "https://www.jobs.nhs.uk/candidate/search/results?keyword=admin&page=1", "maxItems": 100, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"] } }' ``` *** ### 🎯 Use Cases | Industry / Use Case | How the **NHS Job Scraper** Helps | |---------------------|-----------------------------------| | **Labour market analytics** | Track NHS hiring trends by region, band, speciality, and contract type. | | **Recruitment agencies** | Enrich candidate matching with real‑time vacancy data from NHS Trusts. | | **Job board aggregation** | Build a secondary job board focused on NHS roles. | | **Academic research** | Study workforce supply in the UK healthcare system. | | **Salary benchmarking** | Extract salary and band information for comparable roles. | | **Data journalism** | Analyse vacancy patterns, closing dates, and geographical distribution. | *** ### ❓ Frequently Asked Questions **1. Do I need an NHS Jobs account?**\ No. The actor uses the public NHS Jobs website. No login required. **2. Why do I need residential proxies?**\ NHS Jobs uses Cloudflare and may block datacenter IPs after many requests. Residential proxies avoid `Access Denied` pages and ensure stable scraping, especially for large runs. **3. How many jobs can I scrape per run?**\ The actor respects `maxItems` (default 2000). NHS Jobs typically returns up to ~1000–2000 results per search. You can increase by using broader keywords or multiple start URLs. **4. What happens if the actor is blocked (CAPTCHA)?**\ It will log “Blocked by website security” and stop. Retry with a different residential proxy IP or reduce the request rate. **5. How does the pay‑per‑event work?**\ Each successful job advert triggers `Actor.charge(event_name='scraped-result')`. You are charged $0.002 per job. Failed requests (e.g., job detail page 404) are **not charged**. **6. What is the checkpoint feature?**\ The actor saves its state (processed URLs, items pushed, current page number) in the key‑value store. If the run stops, the next run will resume from where it left off without re‑scraping already processed jobs. **7. Can I scrape job descriptions that are long?**\ Yes. The `job_summary`, `main_duties_of_the_job`, and `person_specification` fields capture multi‑paragraph content and bullet points. **8. What is the `person_specification` field?**\ It is a structured array where each item has a `category` (e.g., “Qualifications”, “Experience”) and an `essentials` list (sometimes also `desirables`). This is parsed from the “Person Specification” section of the job advert. **9. How accurate is the band detection?**\ The actor first looks for explicit `Band` field. If not present, it tries to extract band from the salary text (e.g., “Band 5” or “Band 6”). Accuracy is very high for most NHS jobs. **10. What is the actor start fee ($0.40) for?**\ It covers the cost of running the actor container, proxy setup, and checkpoint management – even if no jobs are found. It is a fixed cost per run. *** ### 📝 Technical Notes - **Libraries used:** `curl_cffi` (TLS fingerprinting), `beautifulsoup4` (lxml parser). - **Proxy:** Uses Apify’s `create_proxy_configuration()`. Set `useApifyProxy: true` and choose `RESIDENTIAL` group. - **Concurrency:** Fetches job detail pages in parallel using `asyncio.gather()` for speed. - **Checkpoint:** Stored in key‑value store under `SCRAPER_STATE`. - **Rate limiting:** The actor does not add artificial delays between pages; use proxies to avoid triggering Cloudflare. - **Error handling:** Single job failures do not stop the entire run; they are logged and skipped. *** ### 🔍 SEO Keywords `NHS Job Scraper`, `NHS jobs`, `jobs.nhs.uk scraper`, `NHS vacancy extractor`, `NHS recruitment data`, `UK healthcare jobs`, `NHS salary band scraper`, `NHS job details API`, `Apify NHS actor`, `NHS job advert parser`, `NHS Trust vacancies`, `NHS job search automation` *** ### 🔗 Related Actors - **[UK Job Scraper – Indeed / Reed](https://apify.com/your-username/uk-job-scraper)** – Multi‑site job board extraction. - **[LinkedIn Jobs Scraper](https://apify.com/your-username/linkedin-jobs-scraper)** – Extract corporate job listings. - **[Government Tenders Scraper](https://apify.com/your-username/gov-tenders-scraper)** – UK public sector procurement notices. *** *** **Start scraping **NHS Job Scraper** today – $0.40 per run + $2 per 1,000 successful jobs. Residential proxy ready.** ```` # Actor input Schema ## `startUrl` (type: `string`): Enter the URL of the NHS jobs search results page. ## `maxItems` (type: `integer`): Maximum number of job listings to scrape. ## `proxyConfiguration` (type: `object`): Select proxies to be used by your scraper to prevent blocking. ## Actor input object example ```json { "startUrl": "https://www.jobs.nhs.uk/candidate/search/results?keyword=agency&location=london&language=en&skipPhraseSuggester=true", "maxItems": 2000, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] } } ```` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "startUrl": "https://www.jobs.nhs.uk/candidate/search/results?keyword=agency&location=london&language=en&skipPhraseSuggester=true", "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] } }; // Run the Actor and wait for it to finish const run = await client.actor("scrapepilot/nhs-uk-public-job-board-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "startUrl": "https://www.jobs.nhs.uk/candidate/search/results?keyword=agency&location=london&language=en&skipPhraseSuggester=true", "proxyConfiguration": { "useApifyProxy": True, "apifyProxyGroups": ["RESIDENTIAL"], }, } # Run the Actor and wait for it to finish run = client.actor("scrapepilot/nhs-uk-public-job-board-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrl": "https://www.jobs.nhs.uk/candidate/search/results?keyword=agency&location=london&language=en&skipPhraseSuggester=true", "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] } }' | apify call scrapepilot/nhs-uk-public-job-board-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=scrapepilot/nhs-uk-public-job-board-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "NHS Jobs Scraper — Job Listings & Career Data Extractor", "description": "Scrape NHS job listings from jobs.nhs.uk. Get title, employer, salary, NHS band, location, contract type, closing date, job description, main duties, and structured person specification. Auto-pagination. Checkpoint resume. $2 per 1,000 jobs.", "version": "0.0", "x-build-id": "fJ3N2hFeLbBBF6RL6" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/scrapepilot~nhs-uk-public-job-board-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-scrapepilot-nhs-uk-public-job-board-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/scrapepilot~nhs-uk-public-job-board-scraper/runs": { "post": { "operationId": "runs-sync-scrapepilot-nhs-uk-public-job-board-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/scrapepilot~nhs-uk-public-job-board-scraper/run-sync": { "post": { "operationId": "run-sync-scrapepilot-nhs-uk-public-job-board-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "startUrl" ], "properties": { "startUrl": { "title": "Start Search URL", "type": "string", "description": "Enter the URL of the NHS jobs search results page." }, "maxItems": { "title": "Max Items", "type": "integer", "description": "Maximum number of job listings to scrape.", "default": 2000 }, "proxyConfiguration": { "title": "Proxy Configuration", "type": "object", "description": "Select proxies to be used by your scraper to prevent blocking." } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```