CWjobs UK Tech Jobs Scraper

Scrape UK tech / IT job postings from cwjobs.co.uk — title, employer + logo, location with lat/lng, parsed salary band (min / max / currency / period), datePosted, validThrough, employment type, industry, and directApply flag. One flat row per job from rich JobPosting JSON-LD.

How it works

Why this actor

CWjobs is the tech-vertical UK job board under the StepStone group (same backend as TotalJobs, but pre-filtered to IT / software / engineering / data roles). It's server-rendered HTML with full JobPosting JSON-LD on every detail page — no Akamai, no Cloudflare, no challenge JS. Standard Apify Residential GB returns 200 OK on the first request.

The actor solves what a casual curl can't:

• Apify Residential GB with per-request session rotation — handles transient proxy CONNECT failures and 5xx retries cleanly
• JobPosting JSON-LD extraction — every detail page ships 14 fields server-side; no DOM walking needed for the core data
• Parsed salary band — JSON-LD baseSalary is often missing, so we regex the visible HTML (£70,967 to £83,926 per annum → {min: 70967, max: 83926, currency: "GBP", period: "annum"})
• Hostname-normalised URLs — cwjobs listing pages link to totaljobs.com (sister site, same backend); we rewrite to cwjobs.co.uk for consistency
• Auto-pagination — follows rel="next" or ?page=N until maxItems rows are emitted for that listing URL (each listing gets its own budget)
• Mixed input — listing URLs auto-paginate + emit one row per detail; direct detail URLs scrape one row each

Use cases

• Tech recruitment market intelligence — salary benchmarking for engineering, IT, data, and software roles across UK regions
• Sales prospecting for B2B SaaS — find tech companies hiring in your target verticals + locations
• HR competitive analysis — compare your engineering / data salary bands against UK tech market signal
• ATS / job-aggregator integration — clean structured input for downstream pipelines, with employer details and geo-coordinates
• Geospatial analytics — every row carries location.lat + location.lng from JSON-LD PostalAddress.geo

Input

Field	Type	Required	Notes
startUrls	string[]	yes	Mix of listing URLs (https://www.cwjobs.co.uk/jobs/javascript, /jobs/{keyword}/in-{location}, /jobs/it, /jobs/in-remote) and direct detail URLs (/job/{title-slug}/{org-slug}-job{id}).
maxItems	integer	no	Maximum job rows emitted per listing URL. 3 listings × maxItems: 100 → up to 300 total rows. Direct detail URLs always emit 1 row each. Each row = one paid outputrecord. Default 1000. Free-tier users have a hidden global ceiling of 100 rows.
maxConcurrency	integer	no	Parallel HTTP requests for detail-page fetches. CWjobs is open (no anti-bot), so concurrency 4–8 is comfortable. Default 4.
maxRequestRetries	integer	no	Per-URL retry budget on transient network errors and 5xx responses. Each retry rotates the proxy session with mild exponential backoff. Default 3.
proxy	object	no	Apify Residential GB recommended. CWjobs returns 200 from UK residential IPs on the first attempt; datacenter IPs occasionally see proxy CONNECT failures.

Example input

{
  "startUrls": [
    "https://www.cwjobs.co.uk/jobs/javascript/in-london",
    "https://www.cwjobs.co.uk/jobs/devops/in-manchester",
    "https://www.cwjobs.co.uk/jobs/in-remote"
  ],
  "maxItems": 200,
  "maxConcurrency": 4,
  "proxy": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"], "apifyProxyCountry": "GB" }
}

Output schema

Every row has rowType: "job". 14 fields from JSON-LD + parsed salary band + structured location.

{
  "rowType":           "job",
  "listingUrl":        "https://www.cwjobs.co.uk/jobs/javascript/in-london",
  "jobId":             "107268899",                       // numeric — stable identifier
  "jobUrl":            "https://www.cwjobs.co.uk/job/senior-front-end-engineer-javascript-angular-saas/client-server-job107268899",
  "title":             "Senior Front End Engineer JavaScript Angular SaaS",

  // ── JobPosting JSON-LD ──
  "description":       "<p>Senior Front End Engineer / Developer (JavaScript Angular SaaS) …</p>",  // HTML
  "datePosted":        "2026-04-07T16:34:16.273Z",          // ISO 8601
  "validThrough":      "2026-05-19T16:34:16.273Z",
  "employmentType":    "FULL_TIME",                         // or null
  "industry":          "IT, IT-Web Development",
  "directApply":       true,
  "jobLocationType":   "TELECOMMUTE",                       // null for non-remote
  "applicantLocationRequirements": [{ "type": "Country", "name": "gb" }],

  // ── Employer (from JSON-LD hiringOrganization) ──
  "employer": {
    "name":            "Client Server",
    "url":             "https://www.totaljobs.com/jobs/client-server?cmpId=1375834&cmp=1",
    "logoUrl":         "https://www.totaljobs.com/CompanyLogos/f5cd0600624845eb9e8e49df64db01fc.png"
  },

  // ── Location (from JSON-LD jobLocation.address + geo) ──
  "location": {
    "text":            "Reigate, Surrey, RH2, GB",
    "locality":        "Reigate",
    "region":          "Surrey",
    "postalCode":      "RH2",
    "country":         "GB",
    "lat":             51.24455,
    "lng":             -0.19203
  },

  // ── Salary (parsed from body text via regex — not in JSON-LD) ──
  "salary": {
    "rawText":         "£80K - £85K",
    "min":             80000,
    "max":             85000,
    "currency":        "GBP",                               // ISO code (GBP/USD/EUR)
    "period":          null                                 // "annum" / "hour" / "day" / "week" / "month"
  },

  // ── Apply flow ──
  "applyUrl":          null,                                // JS-resolved at click — see Notes below
  "applyType":         "internal",                          // "internal" / "external" / "unknown" (from directApply)

  "scrapedAt":         "2026-05-15T10:57:23.012Z"
}

Pipeline

1. Classify input — listings (/jobs/...) vs. details (/job/{slug}-job{id}). Listings auto-paginate; details scrape one row each.
2. Fetch via Apify Residential GB — impit with Firefox TLS fingerprint, racing impit + gotScraping + impers (libcurl FFI) under a sliding-window scheduler. CWjobs is open so most calls succeed on the first attempt.
3. For listings: collect detail-URL anchors per page (CWjobs links to totaljobs.com — we normalise to cwjobs.co.uk), follow rel="next" until empty or cap. Then concurrent detail fetches via sliding window.
4. For each detail: parse JobPosting JSON-LD for 14 fields. Run salary regex on body text. Emit one flat row.

⚠️ Apply URL limitation

CWjobs (and sister site TotalJobs) renders the apply button server-side as a disabled placeholder. The real apply URL is loaded async via XHR after click, triggered by JS event listeners we can't execute without a browser.

We tested 15 standard REST-style apply-API endpoints (e.g. /api/applicationredirect/{id}, /api/v1/listings/{id}/apply) — all returned 404. The apply data lives in inline JS state but contains only listingId + listingGlobalId, not the resolved external URL.

v1 ships applyUrl: null and applyType derived from JSON-LD's directApply flag:

• applyType: "internal" → CWjobs hosts the application (directApply: true)
• applyType: "external" → External recruiter hosts (directApply: false)
• applyType: "unknown" → JSON-LD didn't specify

Buyers needing the actual recruiter URL should open the jobUrl in a browser and click apply — or wait for v1.1. No competing CWjobs scraper on Apify Store currently provides applyUrl either — this is a CWjobs-platform limitation, not a scraper gap.

Notes & limitations

• Apify Residential GB recommended. CWjobs returns 200 from UK residential IPs on every probe we tested; non-GB or datacenter IPs occasionally see proxy CONNECT failures.
• Per-listing-page result count varies. Some listing pages return 25 detail anchors, some fewer (CWjobs hides duplicate-job rows). The pagination loop handles empty pages gracefully and continues to page N+1.
• employer.url and employer.logoUrl point to totaljobs.com. This is genuine StepStone-network data — both sister sites share the same employer CMS. Logos and company-jobs URLs work normally on either domain.
• employmentType fill ≈ 70%. Not every JobPosting JSON-LD declares it. We don't synthesize when missing.
• jobLocationType only set for remote/hybrid roles. Non-remote jobs leave it null — matches JSON-LD semantics.
• Salary fill ≈ 100% when surfaced. When a job has a published salary it parses cleanly; when it doesn't (small fraction of jobs) the salary field is null.
• /jobs-at/{company-slug} URLs are not supported in v1 — they redirect to a filtered listing without dedicated org schema. Use the filtered listing path directly (e.g. /jobs/client-server?cmp=1) if you need company-specific results.

FAQ

Which CWjobs URLs work? Two types: listing URLs (/jobs/in-london, /jobs/javascript, /jobs/javascript/in-london, /jobs/it, /jobs/in-remote) which auto-paginate and emit one row per linked job, and direct detail URLs (/job/{title-slug}/{org-slug}-job{id}) which scrape one row each. You can mix both in the same startUrls array.

Why do I need Apify Residential GB? You don't strictly — CWjobs is open and accepts most IPs — but Apify Residential GB is the most reliable pool. Non-GB residentials occasionally see proxy CONNECT failures; UK residential IPs return clean 200s on every probe.

Why is applyUrl always null? CWjobs renders the apply button server-side as a disabled placeholder — the real URL loads async via XHR after click. We can't execute that JS without a browser. The applyType field (derived from JSON-LD's directApply flag) tells you whether the application is hosted internally (directApply: true) or by an external recruiter (directApply: false). See the ⚠️ Apply URL section above for the full breakdown.

Why does the employer URL point at totaljobs.com? CWjobs and TotalJobs are sister sites under StepStone, sharing the same employer CMS. The JSON-LD hiringOrganization.url points at the canonical employer page (totaljobs.com/jobs/{company-slug}), which works as a normal URL. The job URL itself (jobUrl) is normalised to cwjobs.co.uk for consistency.

What does each outputrecord charge cover? One job row with all 14 JSON-LD fields plus the parsed salary band (min/max/currency/period) and structured location (lat/lng). maxItems is per listing URL, so a maxItems: 100 run with 2 listings = up to 200 charges.

Can the parsed salary handle annual / hourly / ranges? Yes. The regex catches £70,967 to £83,926 per annum, £50k - 70k per year, £15 per hour, and single values. The period field normalizes to annum/hour/day/week/month. When a job has no published salary, salary is null — we don't synthesize.

Why does one listing page sometimes return 7 jobs and another 25? CWjobs paginates 25 jobs per page when full but hides duplicate-job rows. The pagination loop handles empty / short pages gracefully and continues to page N+1 until either empty or maxItems rows have been emitted for that listing.

Support

• Bugs / feature requests — open an issue on the GitHub repo
• Custom exports / tailored fields — drop a note via the Apify Store contact form
• Other actors — see my Apify Store profile for the rest of the catalog

---

⚠️ Disclaimer

This Actor is an independent tool and is not affiliated with, endorsed by, or sponsored by cwjobs.co.uk, TotalJobs, StepStone Group, or any of their subsidiaries. All trademarks mentioned are the property of their respective owners.

The scraper extracts only publicly visible job postings rendered server-side by CWjobs — no login, no CAPTCHA solving, no API-key forgery, no private-endpoint probing. The actor honours robots.txt and rate-limits via concurrency cap (default 4) to avoid burdening CWjobs's infrastructure.

Users are responsible for:

• Complying with cwjobs.co.uk's Terms of Service
• Following UK GDPR + your jurisdiction's data-protection laws when storing or processing scraped postings
• Not contacting candidates listed by employers in scraped postings
• Not republishing scraped data in a way that competes commercially with CWjobs

---

SEO Keywords

cwjobs scraper, scrape cwjobs, cwjobs uk scraper, cwjobs.co.uk scraper, cwjobs api, Apify cwjobs, uk tech jobs scraper, uk it jobs scraper, uk software jobs scraper, uk developer jobs scraper, javascript jobs uk, devops jobs uk, data engineer jobs uk, uk job board scraping, jobpostings api, jobposting json-ld scraper, uk recruitment api, recruitment scraper uk, uk tech salary data, salary band extraction, uk tech market data, hiring intelligence uk, employer hiring data, b2b sales prospecting uk, london tech jobs scraping, manchester tech jobs scraper, remote uk jobs scraper, stepstone uk jobs, apify residential gb