# UK Civil Service Jobs Scraper (`automation-lab/uk-civil-service-jobs-scraper`) Actor

Search UK Civil Service vacancies and extract structured salaries, grades, locations, deadlines, job descriptions, benefits, application guidance, and contacts.

- **URL**: https://apify.com/automation-lab/uk-civil-service-jobs-scraper.md
- **Developed by:** [Stas Persiianenko](https://apify.com/automation-lab) (community)
- **Categories:** Jobs
- **Stats:** 3 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per event

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## UK Civil Service Jobs Scraper

Search current vacancies on the official UK Civil Service Jobs website and turn each advert into a structured, integration-ready record.

The Actor searches by keywords and location, follows result pages, and enriches every match from its vacancy page.
It can also process specific Civil Service Jobs URLs.
No login, browser, or user-supplied proxy is required.

### What does UK Civil Service Jobs Scraper do?

The Actor automates three steps:

1. Search the public Civil Service Jobs vacancy catalogue.
2. Paginate and deduplicate matching adverts.
3. Extract full job details into the default Apify dataset.

Records include salary ranges, grade, department, location, contract, business area, role type, working pattern, deadline, full advert text, benefits, application guidance, and applicant contacts.

### Who is it for?

- **Recruiters and job boards** refreshing public-sector vacancies.
- **Labor-market analysts** comparing salary, grade, department, and location trends.
- **Alert products** monitoring newly listed or soon-closing roles.
- **Career services** building searchable Civil Service vacancy feeds.
- **Data teams** loading public vacancy records into warehouses and dashboards.

The stable `reference` and `vacancyId` fields make scheduled-run deduplication straightforward.

### Why use this Actor?

- Uses the official public vacancy source.
- Supports both search and direct vacancy URL enrichment.
- Returns full detail text, not only result-card summaries.
- Produces stable canonical vacancy URLs.
- Uses lightweight HTTP requests rather than a browser.
- Handles the site's public proof-of-work session gate automatically.
- Limits concurrency to protect source reliability.

### What data can it extract?

| Field | Meaning |
|---|---|
| `title` | Vacancy title |
| `department` | Hiring department or public body |
| `reference` | Published Civil Service vacancy reference |
| `vacancyId` | Internal stable vacancy identifier |
| `locations` | Advertised work locations |
| `salary` | Human-readable published salary |
| `salaryMin`, `salaryMax` | Parsed numeric annual salary bounds |
| `currency` | Published currency code, normally GBP |
| `grade` | Civil Service job grade |
| `contractType` | Permanent, fixed term, or another contract type |
| `businessArea` | Department business area |
| `roleTypes` | Published role categories |
| `workingPatterns` | Full-time, part-time, flexible, job share, and similar patterns |
| `numberOfJobs` | Number of posts when published |
| `closingTime` | Published application deadline |
| `jobSummary` | Job summary text |
| `jobDescription` | Full job description |
| `personSpecification` | Candidate requirements |
| `benefits` | Benefits and pension information |
| `applicationInstructions` | Selection and application guidance |
| `contactName` | Vacancy contact name |
| `contactEmail` | Vacancy contact email |
| `contactTelephone` | Vacancy contact telephone |
| `recruitmentEmail` | Recruitment team email |
| `url` | Stable canonical vacancy URL |
| `scrapedAt` | ISO timestamp for extraction |

Fields not published by a particular advert are returned as `null` or an empty array.

### How to get started

1. Open the Actor in Apify Console.
2. Enter keywords such as `data scientist`, `policy`, or `cyber security`.
3. Optionally enter a UK town, region, or postcode.
4. Choose the maximum number of vacancies.
5. Click **Start**.
6. Open the **Vacancies** dataset view.
7. Export records as JSON, CSV, Excel, XML, or RSS.

For one advert, clear `query` and add its URL under `startUrls`.

### Input parameters

| Parameter | Type | Default | Description |
|---|---|---:|---|
| `query` | string | `data scientist` | Title, skill, or keyword search; maximum 99 characters |
| `location` | string | empty | UK town, region, or postcode |
| `startUrls` | array | empty | Civil Service Jobs vacancy URLs to enrich |
| `maxItems` | integer | 25 | Maximum unique records, from 1 to 1,000 |
| `maxConcurrency` | integer | 3 | Parallel detail requests, from 1 to 5 |

At least one non-empty `query` or one valid `startUrls` entry is required.
Only HTTPS URLs on `www.civilservicejobs.service.gov.uk` are accepted.

### Input examples

Search all locations:

```json
{
  "query": "data",
  "location": "",
  "maxItems": 25,
  "maxConcurrency": 3
}
```

Search in London:

```json
{
  "query": "policy",
  "location": "London",
  "maxItems": 100
}
```

Enrich one vacancy:

```json
{
  "query": "",
  "startUrls": [
    { "url": "https://www.civilservicejobs.service.gov.uk/csr/jobs.cgi?jcode=2006486" }
  ],
  "maxItems": 1
}
```

### Output example

A current result has this shape:

```json
{
  "title": "Data Scientist (Industrialisation)",
  "department": "HM Revenue and Customs",
  "reference": "470492",
  "vacancyId": "2006486",
  "locations": [
    "Leeds Regional Centre - Wellington Place",
    "Manchester Regional Centre - Three New Bailey"
  ],
  "salary": "£45,544 - £49,523",
  "salaryMin": 45544,
  "salaryMax": 49523,
  "currency": "GBP",
  "grade": "Senior Executive Officer",
  "contractType": "Permanent",
  "workingPatterns": ["Flexible working", "Full-time", "Job share", "Part-time"],
  "closingTime": "11:55 pm on Wednesday 5th August 2026",
  "url": "https://www.civilservicejobs.service.gov.uk/csr/jobs.cgi?jcode=2006486",
  "scrapedAt": "2026-07-26T00:00:00.000Z"
}
```

Long text fields are shortened here only for readability.
The dataset contains their complete extracted values.

### How much does it cost to scrape UK Civil Service jobs?

Pay-per-event pricing consists of a **$0.005 Actor start** and one charge for each saved vacancy.
The current FREE-tier item price is **$0.000041071 per vacancy**, with lower unit prices on higher Apify pricing tiers.

Examples on the FREE tier:

| Saved vacancies | Start | Items | Total |
|---:|---:|---:|---:|
| 10 | $0.005 | $0.000411 | $0.005411 |
| 100 | $0.005 | $0.004107 | $0.009107 |
| 1,000 | $0.005 | $0.041071 | $0.046071 |

Only successfully saved records trigger item charges.
No-result searches do not trigger item charges.
Apify rounds and bills according to the active pricing shown in Console.

### Scheduling and deduplication

Run the Actor daily or hourly with an Apify schedule.
Use `reference` as the source-facing vacancy key.
Use `vacancyId` or `url` as a second stable key.
Compare `scrapedAt` and `closingTime` to update downstream indexes or alerts.

A typical workflow is:

1. Schedule a keyword/location search.
2. Export the dataset through an integration or webhook.
3. Upsert by `reference`.
4. Notify only when a reference is new or its deadline changed.
5. Remove or archive records after their closing time.

### Integration ideas

- Send new roles to Slack or Microsoft Teams.
- Load salary and grade fields into BigQuery or Snowflake.
- Publish selected records to a recruitment site.
- Create daily email alerts by department and location.
- Track hiring volume by business area over time.
- Feed vacancy text into a search or matching system.
- Trigger Zapier or Make workflows when runs finish.

### Run with the Apify API using cURL

```bash
curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~uk-civil-service-jobs-scraper/runs?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"query":"policy","location":"London","maxItems":25}'
```

To wait and receive dataset items:

```bash
curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~uk-civil-service-jobs-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"query":"data scientist","maxItems":10}'
```

### Run with JavaScript

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/uk-civil-service-jobs-scraper').call({
  query: 'data scientist',
  location: '',
  maxItems: 25,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);
```

### Run with Python

```python
import os
from apify_client import ApifyClient

client = ApifyClient(os.environ['APIFY_TOKEN'])
run = client.actor('automation-lab/uk-civil-service-jobs-scraper').call(
    run_input={'query': 'policy', 'location': 'London', 'maxItems': 25}
)
items = client.dataset(run['defaultDatasetId']).list_items().items
print(items)
```

### Use with Apify MCP

Add this Actor to Claude Code:

```bash
claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/uk-civil-service-jobs-scraper"
```

Claude Desktop, Cursor, and VS Code configuration:

```json
{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=automation-lab/uk-civil-service-jobs-scraper"
    }
  }
}
```

Example prompts:

- “Find 20 current UK Civil Service data scientist vacancies and rank them by minimum salary.”
- “Collect London policy jobs and return reference, department, grade, deadline, and URL.”
- “Enrich this Civil Service Jobs URL and summarize the person specification.”

### Reliability and source behavior

Civil Service Jobs uses a public computational proof-of-work gate.
The Actor solves it automatically inside a cookie-preserving HTTP session.
A normal run therefore needs no human CAPTCHA, browser, login, or proxy setting.

Requests use bounded retries for temporary network failures, HTTP 429, and upstream 5xx responses.
Malformed inputs, unsupported domains, and changed page structures fail clearly instead of returning misleading empty success.

### Limits and tips

- Keep `maxConcurrency` at 3 unless a controlled test justifies changing it.
- Use narrower keywords for more relevant output.
- Location matching follows the source website's own search behavior.
- Search result order follows the source relevance order.
- A vacancy may omit optional contacts or salary bounds.
- Closed adverts can disappear, so store records you need historically.
- The Actor extracts public advert data; it does not submit applications.
- Maximum output is 1,000 records per run.

### No-result searches

A valid query can naturally return no current vacancies.
In that case the run succeeds with an empty dataset and no item charges.
Try a broader keyword, remove the location, or run again later.
A challenge page or unrecognized HTML is treated as an error, not as zero results.

### Troubleshooting

**Why did my direct URL fail?**

Confirm that it is an HTTPS vacancy URL on `www.civilservicejobs.service.gov.uk` and that the advert is still available.
Search pages and unrelated support pages are intentionally rejected.

**Why are some fields null?**

Departments choose which optional information to publish.
The Actor does not invent missing salary, contact, or classification values.

**Why did the run fail after retries?**

The source may be temporarily unavailable or rate-limiting requests.
Retry later with the default concurrency.
If the source changed its public challenge or HTML, share the run ID with support.

### Responsible use and legality

This Actor extracts publicly accessible vacancy advertisements.
You are responsible for complying with applicable laws, website terms, database rights, and your organization's policies.
Do not use contact details for spam or unlawful profiling.
Collect only data needed for a legitimate purpose, retain it only as necessary, and respect deletion or correction obligations.
This documentation is not legal advice.

### FAQ

**Does it need a Civil Service Jobs account?**

No. It uses anonymous public search and advert pages.

**Does it use residential proxies?**

No. Version 0.1 uses direct HTTP and exposes no automatic proxy fallback.

**Can it apply for jobs?**

No. It extracts vacancy data only.

**Can I scrape a single advert?**

Yes. Clear `query` and provide the vacancy under `startUrls`.

**Can I export CSV or Excel?**

Yes. Use the dataset export controls or dataset API.

**Can I monitor changes?**

Yes. Schedule repeat runs and upsert records by `reference` or `vacancyId`.

### Support

For reproducible help, include the Apify run ID, non-secret input, approximate run time, and expected vacancy or query.
Do not post API tokens or private downstream credentials.

### Related automation-lab Actors

This Actor is intentionally source-specific and currently works as a standalone Civil Service vacancy feed.
After publication, complementary automation-lab job-search Actors may be linked here when they offer a genuine multi-source recruitment workflow.

# Actor input Schema

## `query` (type: `string`):

Job title, skill, or keywords to search, for example data scientist or policy.

## `location` (type: `string`):

Optional UK town, region, or postcode. Leave blank to search all locations.

## `startUrls` (type: `array`):

Optional Civil Service Jobs vacancy URLs to enrich alongside, or instead of, a keyword search.

## `maxItems` (type: `integer`):

Maximum number of unique vacancy records to save.

## `maxConcurrency` (type: `integer`):

Number of vacancy detail pages fetched in parallel. Keep the default for reliable source access.

## Actor input object example

```json
{
  "query": "data scientist",
  "startUrls": [],
  "maxItems": 20,
  "maxConcurrency": 3
}
```

# Actor output Schema

## `overview` (type: `string`):

Open or download the default dataset of enriched vacancy records.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "query": "data scientist",
    "location": "",
    "startUrls": [],
    "maxItems": 20,
    "maxConcurrency": 3
};

// Run the Actor and wait for it to finish
const run = await client.actor("automation-lab/uk-civil-service-jobs-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "query": "data scientist",
    "location": "",
    "startUrls": [],
    "maxItems": 20,
    "maxConcurrency": 3,
}

# Run the Actor and wait for it to finish
run = client.actor("automation-lab/uk-civil-service-jobs-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "query": "data scientist",
  "location": "",
  "startUrls": [],
  "maxItems": 20,
  "maxConcurrency": 3
}' |
apify call automation-lab/uk-civil-service-jobs-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,automation-lab/uk-civil-service-jobs-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/RlcolAzdwUOQozfbh/builds/OsUvkYwhdIwk2rl5R/openapi.json