UK Civil Service Jobs Scraper

Search current vacancies on the official UK Civil Service Jobs website and turn each advert into a structured, integration-ready record.

The Actor searches by keywords and location, follows result pages, and enriches every match from its vacancy page. It can also process specific Civil Service Jobs URLs. No login, browser, or user-supplied proxy is required.

What does UK Civil Service Jobs Scraper do?

The Actor automates three steps:

1. Search the public Civil Service Jobs vacancy catalogue.
2. Paginate and deduplicate matching adverts.
3. Extract full job details into the default Apify dataset.

Records include salary ranges, grade, department, location, contract, business area, role type, working pattern, deadline, full advert text, benefits, application guidance, and applicant contacts.

Who is it for?

• Recruiters and job boards refreshing public-sector vacancies.
• Labor-market analysts comparing salary, grade, department, and location trends.
• Alert products monitoring newly listed or soon-closing roles.
• Career services building searchable Civil Service vacancy feeds.
• Data teams loading public vacancy records into warehouses and dashboards.

The stable reference and vacancyId fields make scheduled-run deduplication straightforward.

Why use this Actor?

• Uses the official public vacancy source.
• Supports both search and direct vacancy URL enrichment.
• Returns full detail text, not only result-card summaries.
• Produces stable canonical vacancy URLs.
• Uses lightweight HTTP requests rather than a browser.
• Handles the site's public proof-of-work session gate automatically.
• Limits concurrency to protect source reliability.

What data can it extract?

Field	Meaning
title	Vacancy title
department	Hiring department or public body
reference	Published Civil Service vacancy reference
vacancyId	Internal stable vacancy identifier
locations	Advertised work locations
salary	Human-readable published salary
salaryMin, salaryMax	Parsed numeric annual salary bounds
currency	Published currency code, normally GBP
grade	Civil Service job grade
contractType	Permanent, fixed term, or another contract type
businessArea	Department business area
roleTypes	Published role categories
workingPatterns	Full-time, part-time, flexible, job share, and similar patterns
numberOfJobs	Number of posts when published
closingTime	Published application deadline
jobSummary	Job summary text
jobDescription	Full job description
personSpecification	Candidate requirements
benefits	Benefits and pension information
applicationInstructions	Selection and application guidance
contactName	Vacancy contact name
contactEmail	Vacancy contact email
contactTelephone	Vacancy contact telephone
recruitmentEmail	Recruitment team email
url	Stable canonical vacancy URL
scrapedAt	ISO timestamp for extraction

Fields not published by a particular advert are returned as null or an empty array.

How to get started

1. Open the Actor in Apify Console.
2. Enter keywords such as data scientist, policy, or cyber security.
3. Optionally enter a UK town, region, or postcode.
4. Choose the maximum number of vacancies.
5. Click Start.
6. Open the Vacancies dataset view.
7. Export records as JSON, CSV, Excel, XML, or RSS.

For one advert, clear query and add its URL under startUrls.

Input parameters

Parameter	Type	Default	Description
query	string	data scientist	Title, skill, or keyword search; maximum 99 characters
location	string	empty	UK town, region, or postcode
startUrls	array	empty	Civil Service Jobs vacancy URLs to enrich
maxItems	integer	25	Maximum unique records, from 1 to 1,000
maxConcurrency	integer	3	Parallel detail requests, from 1 to 5

At least one non-empty query or one valid startUrls entry is required. Only HTTPS URLs on www.civilservicejobs.service.gov.uk are accepted.

Input examples

Search all locations:

{
  "query": "data",
  "location": "",
  "maxItems": 25,
  "maxConcurrency": 3
}

Search in London:

{
  "query": "policy",
  "location": "London",
  "maxItems": 100
}

Enrich one vacancy:

{
  "query": "",
  "startUrls": [
    { "url": "https://www.civilservicejobs.service.gov.uk/csr/jobs.cgi?jcode=2006486" }
  ],
  "maxItems": 1
}

Output example

A current result has this shape:

{
  "title": "Data Scientist (Industrialisation)",
  "department": "HM Revenue and Customs",
  "reference": "470492",
  "vacancyId": "2006486",
  "locations": [
    "Leeds Regional Centre - Wellington Place",
    "Manchester Regional Centre - Three New Bailey"
  ],
  "salary": "£45,544 - £49,523",
  "salaryMin": 45544,
  "salaryMax": 49523,
  "currency": "GBP",
  "grade": "Senior Executive Officer",
  "contractType": "Permanent",
  "workingPatterns": ["Flexible working", "Full-time", "Job share", "Part-time"],
  "closingTime": "11:55 pm on Wednesday 5th August 2026",
  "url": "https://www.civilservicejobs.service.gov.uk/csr/jobs.cgi?jcode=2006486",
  "scrapedAt": "2026-07-26T00:00:00.000Z"
}

Long text fields are shortened here only for readability. The dataset contains their complete extracted values.

How much does it cost to scrape UK Civil Service jobs?

Pay-per-event pricing consists of a $0.005 Actor start and one charge for each saved vacancy. The current FREE-tier item price is $0.000041071 per vacancy, with lower unit prices on higher Apify pricing tiers.

Examples on the FREE tier:

Saved vacancies	Start	Items	Total
10	$0.005	$0.000411	$0.005411
100	$0.005	$0.004107	$0.009107
1,000	$0.005	$0.041071	$0.046071

Only successfully saved records trigger item charges. No-result searches do not trigger item charges. Apify rounds and bills according to the active pricing shown in Console.

Scheduling and deduplication

Run the Actor daily or hourly with an Apify schedule. Use reference as the source-facing vacancy key. Use vacancyId or url as a second stable key. Compare scrapedAt and closingTime to update downstream indexes or alerts.

A typical workflow is:

1. Schedule a keyword/location search.
2. Export the dataset through an integration or webhook.
3. Upsert by reference.
4. Notify only when a reference is new or its deadline changed.
5. Remove or archive records after their closing time.

Integration ideas

• Send new roles to Slack or Microsoft Teams.
• Load salary and grade fields into BigQuery or Snowflake.
• Publish selected records to a recruitment site.
• Create daily email alerts by department and location.
• Track hiring volume by business area over time.
• Feed vacancy text into a search or matching system.
• Trigger Zapier or Make workflows when runs finish.

Run with the Apify API using cURL

curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~uk-civil-service-jobs-scraper/runs?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"query":"policy","location":"London","maxItems":25}'

To wait and receive dataset items:

curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~uk-civil-service-jobs-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"query":"data scientist","maxItems":10}'

Run with JavaScript

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/uk-civil-service-jobs-scraper').call({
  query: 'data scientist',
  location: '',
  maxItems: 25,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Run with Python

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ['APIFY_TOKEN'])
run = client.actor('automation-lab/uk-civil-service-jobs-scraper').call(
    run_input={'query': 'policy', 'location': 'London', 'maxItems': 25}
)
items = client.dataset(run['defaultDatasetId']).list_items().items
print(items)

Use with Apify MCP

Add this Actor to Claude Code:

claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/uk-civil-service-jobs-scraper"

Claude Desktop, Cursor, and VS Code configuration:

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=automation-lab/uk-civil-service-jobs-scraper"
    }
  }
}

Example prompts:

• “Find 20 current UK Civil Service data scientist vacancies and rank them by minimum salary.”
• “Collect London policy jobs and return reference, department, grade, deadline, and URL.”
• “Enrich this Civil Service Jobs URL and summarize the person specification.”

Reliability and source behavior

Civil Service Jobs uses a public computational proof-of-work gate. The Actor solves it automatically inside a cookie-preserving HTTP session. A normal run therefore needs no human CAPTCHA, browser, login, or proxy setting.

Requests use bounded retries for temporary network failures, HTTP 429, and upstream 5xx responses. Malformed inputs, unsupported domains, and changed page structures fail clearly instead of returning misleading empty success.

Limits and tips

• Keep maxConcurrency at 3 unless a controlled test justifies changing it.
• Use narrower keywords for more relevant output.
• Location matching follows the source website's own search behavior.
• Search result order follows the source relevance order.
• A vacancy may omit optional contacts or salary bounds.
• Closed adverts can disappear, so store records you need historically.
• The Actor extracts public advert data; it does not submit applications.
• Maximum output is 1,000 records per run.

No-result searches

A valid query can naturally return no current vacancies. In that case the run succeeds with an empty dataset and no item charges. Try a broader keyword, remove the location, or run again later. A challenge page or unrecognized HTML is treated as an error, not as zero results.

Troubleshooting

Why did my direct URL fail?

Confirm that it is an HTTPS vacancy URL on www.civilservicejobs.service.gov.uk and that the advert is still available. Search pages and unrelated support pages are intentionally rejected.

Why are some fields null?

Departments choose which optional information to publish. The Actor does not invent missing salary, contact, or classification values.

Why did the run fail after retries?

The source may be temporarily unavailable or rate-limiting requests. Retry later with the default concurrency. If the source changed its public challenge or HTML, share the run ID with support.

Responsible use and legality

This Actor extracts publicly accessible vacancy advertisements. You are responsible for complying with applicable laws, website terms, database rights, and your organization's policies. Do not use contact details for spam or unlawful profiling. Collect only data needed for a legitimate purpose, retain it only as necessary, and respect deletion or correction obligations. This documentation is not legal advice.

FAQ

Does it need a Civil Service Jobs account?

No. It uses anonymous public search and advert pages.

Does it use residential proxies?

No. Version 0.1 uses direct HTTP and exposes no automatic proxy fallback.

Can it apply for jobs?

No. It extracts vacancy data only.

Can I scrape a single advert?

Yes. Clear query and provide the vacancy under startUrls.

Can I export CSV or Excel?

Yes. Use the dataset export controls or dataset API.

Can I monitor changes?

Yes. Schedule repeat runs and upsert records by reference or vacancyId.

Support

For reproducible help, include the Apify run ID, non-secret input, approximate run time, and expected vacancy or query. Do not post API tokens or private downstream credentials.

Related automation-lab Actors

This Actor is intentionally source-specific and currently works as a standalone Civil Service vacancy feed. After publication, complementary automation-lab job-search Actors may be linked here when they offer a genuine multi-source recruitment workflow.