Creative Scotland Opportunities Scraper

Scrape opportunities.creativescotland.com — Scotland's creative-sector hub for paid jobs, workshops, funding, residencies, training, and exhibitions. The site sits behind a Cloudflare WAF rule that blocks almost every request from outside the UK. This actor comes with its own UK-pinned residential proxy, walks the full opportunities board page by page, and returns every live listing — 600+ of them. JSON or CSV out, no compute charge per run, just per result.

How it works

✨ Why use this scraper?

Tracking Scotland's creative sector is hard — Creative Scotland's Opportunities tool is the central hub but Cloudflare's WAF makes direct scraping nearly impossible. This actor solves the proxy-rotation problem so you can focus on the data.

• 🛡️ Cloudflare WAF handled, no browser needed. No browser automation, no Camoufox, no cf_clearance cookie capture. The WAF here gates on the exit IP's country, so the built-in proxy is pinned to UK exits — a recent 41-page full-board run passed every page on the first request.
• ⚡ The whole board, not a sample. 600+ live opportunities, 16 per page, walked to the last page automatically. Every row arrives with title, organisation, region, category, deadline, teaser and URL.
• 🎨 Jobs + workshops + funding + residencies in one dataset. Rows carry Creative Scotland's own taxonomy in categories[] — Workshops, Jobs, Funding and Grants, Artist Residencies, Call for Entries, Commissions, Tenders, and nine more — plus a coarse type (job vs opportunity) for quick filtering.
• 📧 Contact email captured. Turn on enrichDetail and each row gains the organiser's contact address as applyEmail, their own website, the full description body and any stated day rate.
• 🇬🇧 Scotland-wide. Edinburgh galleries, Highland community workshops, Glasgow theatres, rural craft schools, festival organisers.
• 📤 Clean exports. One row per opportunity. JSON + CSV exported automatically.

🎯 Use cases

Team	What they build
Creative-sector recruiters	Daily new-vacancy feeds across Scottish arts orgs
Festival / event aggregators	Auto-populate Scotland-wide creative events calendars
Craft schools / workshop platforms	Discover competitor workshop offerings
Funders & cultural strategists	Track who's hiring + what training is being offered in Scottish creative sector
Researchers	Scottish creative labour-market datasets

📥 Supported inputs

URL pattern	Behaviour
https://opportunities.creativescotland.com/	Whole board (default) — every live opportunity, all pages
https://opportunities.creativescotland.com/?page=2	Whole board — the walk always starts at page 1
https://opportunities.creativescotland.com/opportunity/index/<uuid>	Single opportunity — stub + optional detail fetch
https://opportunities.creativescotland.com/api/rss/	Whole board — see the note below
(anything else on the domain)	Routed to the whole board

Leave startUrls empty for the whole board.

Note on /api/rss/. Creative Scotland's RSS feed still responds, but since August 2026 it returns an empty channel with no items in it. That URL is still accepted and still means "the whole board" — the actor just reads the HTML board instead, which is also where the other 600 opportunities the feed never carried live.

⚠️ Proxy requirement (mandatory)

Nothing to configure — the actor ships with its own UK residential proxy at no extra cost to you. You only need the proxy input if you want to route through your own pool instead.

If you do bring your own, it must be residential and UK-exit. Cloudflare on this site blocks by exit-IP country: measured over 20 attempts per combination, UK residential exits passed 80/80 while an untargeted worldwide residential pool passed about 5%. Datacenter proxies and direct requests are blocked instantly regardless of country.

🔄 How it works

1. Resolve start URLs — default is the whole board.
2. Fetch board page N through the UK residential pool:
   • GET with a Firefox-144 UA and realistic accept headers
   • If 200 OK and the body isn't a Cloudflare block page → success
   • Otherwise rotate to a fresh exit and retry, up to cfMaxAttempts
3. Parse the 16 cards on the page — title, opportunity URL, UUID, organisation, region, category, deadline and teaser, read from the page's schema.org microdata.
4. Advance to page N+1 and repeat until a page comes back with no cards, or maxItems is covered.
5. Push one normalised row per opportunity, deduplicated by UUID.
6. Optional: when enrichDetail: true, fetch each opportunity's own page for the full description body, contact email, organisation website, full region list and any stated fee.

⚙️ Input parameters

Parameter	Type	Default	Description
startUrls	array	["https://opportunities.creativescotland.com/"]	Board URL, opportunity URLs, or anything on the domain.
enrichDetail	boolean	false	When true, fetches each opportunity's own page for the full description, contact email, organisation website and fee. One extra request per row.
cfMaxAttempts	integer	8	Max proxy attempts per URL. With the UK-pinned pool the first attempt almost always succeeds; this is the safety net.
postedWithinHours	integer	(none)	Only return recent opportunities. Maps onto Creative Scotland's own date buckets — up to 168 = last week, up to 336 = last 2 weeks, up to 744 = last month. Empty/0 = all.
maxItems	integer	1000	Hard cap on rows pushed. The board carries 600+ live opportunities.
maxConcurrency	integer	3	Parallel detail-page fetch limit (matters only with enrichDetail: true).
maxRequestRetries	integer	5	Reserved — superseded by cfMaxAttempts for this actor.
proxy	object	built-in UK residential	Optional override. Must be residential with UK exits — see the proxy section above.

📊 Output overview

Each scraped opportunity is one single dataset row. Jobs get type: "job"; everything else gets type: "opportunity". The precise label is in categories[0], taken straight from Creative Scotland's own taxonomy: Workshops, Jobs, Professional Development, Call for Entries, Studio Spaces and Facilities, Conferences and Talks, Funding and Grants, Masterclasses, Board, Committee and Trustee roles, Artist Residencies, Commissions, Research, Consultations, Business Support, Awards and Competitions, Tenders.

📦 Output sample

Real row from a run with enrichDetail: true (long text abbreviated):

{
  "type": "job",
  "source": "opportunities.creativescotland.com",
  "jobId": "b547b093-ef22-442b-8c8d-03d4edcda5c2",
  "slug": "b547b093-ef22-442b-8c8d-03d4edcda5c2",
  "jobUrl": "https://opportunities.creativescotland.com/opportunity/index/b547b093-ef22-442b-8c8d-03d4edcda5c2/",
  "title": "Events Management & Marketing Trainer",
  "companyName": "EveryStage",
  "companyWebsite": "https://everystage.uk/",
  "companyDomain": "everystage.uk",
  "location": "East Ayrshire, East Dunbartonshire, East Lothian, East Renfrewshire, Edinburgh City, Falkirk, Fife, Glasgow City, Inverclyde, Midlothian, Perth and Kinross, Stirling, West Lothian",
  "remote": false,
  "salary": null,
  "salaryRaw": "£140 - £160 per day",
  "categories": [
    "Jobs"
  ],
  "employmentTypes": [],
  "contractType": null,
  "hours": null,
  "postedDate": null,
  "closingDate": "2026-08-28T17:00:00.000Z",
  "modifiedDate": null,
  "description": "Events Management &amp; Marketing Trainer (Freelance)<br>Loc… (full HTML)",
  "descriptionText": "Events Management & Marketing Trainer (Freelance)\nLocations: Glasgow & Edinburgh (potential future opportuniti…",
  "applyType": "email",
  "applyUrl": "https://opportunities.creativescotland.com/opportunity/index/b547b093-ef22-442b-8c8d-03d4edcda5c2/",
  "applyEmail": "cairistiona@everystage.uk",
  "externalApplyUrl": null,
  "scrapedAt": "2026-08-19T04:24:31.536Z",
  "opportunityId": "b547b093-ef22-442b-8c8d-03d4edcda5c2",
  "opportunityUrl": "https://opportunities.creativescotland.com/opportunity/index/b547b093-ef22-442b-8c8d-03d4edcda5c2/",
  "descriptionTeaser": "EveryStage Ltd is looking for a dynamic and qualified Freelance Trainer to deliver our Events Management and Marketing focused employability courses.",
  "fee": "£140 - £160 per day",
  "status": "publish",
  "closingDateRaw": "Deadline: 28 August 2026 at 17:00"
}

🗂 Key output fields

Group	Fields
Identifiers	type (job or opportunity), source, opportunityId (UUID), jobId (= opportunityId), slug, jobUrl, opportunityUrl, scrapedAt
Content	title, description (HTML, only when enrichDetail: true), descriptionText (plain), descriptionTeaser (the board's own summary)
Dates	closingDate (ISO deadline, on every row), closingDateRaw (the deadline exactly as the site words it), postedDate (always null — the site publishes none)
Employer / organiser	companyName (on every row), companyWebsite and companyDomain (with enrichDetail)
Categorisation	categories[] — Creative Scotland's own label, e.g. ["Workshops"], ["Funding and Grants"], ["Artist Residencies"]
Location	location — the board's region summary, or the full council-area list with enrichDetail
Apply flow	applyType (email when a contact address was found, else internal), applyUrl (the opportunity page), applyEmail (with enrichDetail)
Optional (with enrichDetail)	full HTML description, applyEmail, companyWebsite, fee / salaryRaw (e.g. "£140 - £160 per day"), full region list

❓ FAQ

What if I see "CF rotation exhausted"? Cloudflare here blocks by exit-IP country, and the built-in proxy is pinned to UK exits, so this should be rare — a recent full-board walk cleared all 41 pages on the first attempt each. If you overrode proxy with your own pool, check it is residential and UK-targeted; that is almost always the cause. Otherwise raise cfMaxAttempts.

Why is enrichDetail off by default? It adds one request per opportunity, so a full 600-row board goes from ~41 requests to ~640. The board itself already gives you title, organisation, region, category, deadline, teaser and URL. Turn detail on when you need the contact email, the full description body, the organisation's website or the day rate.

Why does type say opportunity when it's clearly a workshop? type is a coarse job/not-job flag. The categories[] array carries Creative Scotland's real label — Workshops, Funding and Grants, Artist Residencies, and so on. Filter on categories[0] for the specific shape.

Why is postedDate always null? Creative Scotland does not publish a posting date, on the board or on the opportunity pages — only a deadline. That is why postedWithinHours maps onto the site's own last-week / last-2-weeks / last-month buckets rather than filtering on an exact timestamp.

Can I scrape private pages or authenticated content? No. Only the public opportunities board and public /opportunity/index/<uuid> pages.

How do I limit results? Set maxItems. The board carries 600+ live opportunities, so leave it at the 1000 default to capture everything.

💬 Support

• For issues or feature requests, please use the Issues tab on the actor's Apify Console page.
• Author's website: https://muhamed-didovic.github.io/
• Email: muhamed.didovic@gmail.com

🛠 Additional services

• Custom output shape, additional fields, or one-off datasets: muhamed.didovic@gmail.com
• Cloudflare-WAF-bypass scrapers for other CF-protected sites in your portfolio (same residential-rotation pattern works on many): drop an email.
• For API access (no Apify fee, just usage): muhamed.didovic@gmail.com

🔎 Explore more scrapers

See other scrapers at memo23's Apify profile — covering job boards, real estate, social media, and more.

---

🤖 For AI Agents & LLM Apps

Compact reference for AI agents calling this actor via the Apify MCP server or the Apify API (actor: memo23/creativescotland-scraper).

Purpose: Scrape Scotland's creative-sector opportunities board (opportunities.creativescotland.com) — jobs, workshops, funding, residencies and more — walking the whole 600+ listing board through a built-in UK-pinned residential proxy that clears the site's Cloudflare WAF.

Minimal input:

{
  "startUrls": ["https://opportunities.creativescotland.com/"],
  "enrichDetail": false,
  "cfMaxAttempts": 8,
  "postedWithinHours": 0,
  "maxItems": 50,
  "maxConcurrency": 3,
  "minConcurrency": 1,
  "maxRequestRetries": 5,
  "proxy": {}
}

Single-opportunity mode: pass "startUrls": ["https://opportunities.creativescotland.com/opportunity/index/<uuid>"].

Output: one dataset row per opportunity — type (job or opportunity), source, jobId, slug, jobUrl, title, companyName, companyWebsite, companyDomain, location, remote, salary, salaryRaw, categories, employmentTypes, contractType, hours, postedDate, closingDate, modifiedDate, description, descriptionText, applyType, applyUrl, applyEmail, externalApplyUrl, scrapedAt, opportunityId, opportunityUrl, descriptionTeaser, fee, status, closingDateRaw.

Behaviors an agent should know:

• startUrls accepts the whole board (default), ?page=N (walk always starts at page 1), /opportunity/index/<uuid> single-opportunity URLs, and the legacy /api/rss/ (now emptied — falls back to the HTML board). Empty = whole board.
• Always set maxItems. The board carries 600+ live opportunities; leave the 1000 default to capture everything, or cap for a sample.
• enrichDetail: false (default) already returns title, organisation, region, category, deadline, teaser and URL. Set true to add one request per row for the full description, applyEmail, companyWebsite, fee and full region list.
• postedWithinHours maps onto the site's own buckets (168 = last week, 336 = 2 weeks, 744 = last month; 0/empty = all) because Creative Scotland publishes no per-opportunity posting date, so postedDate is always null.
• Built-in UK residential proxy is mandatory for this site; any override must be residential with UK exits. cfMaxAttempts (min 5) is the rotation safety net. Public pages only, no CAPTCHA/JS-challenge bypass.
• Billed per result — no per-run compute charge.

---

⚠️ Disclaimer

This Actor is an independent tool and is not affiliated with, endorsed by, or sponsored by Creative Scotland, Alba Chruthachail, or any of their subsidiaries or affiliates. All trademarks mentioned are the property of their respective owners.

The scraper accesses only the publicly available opportunities board and public opportunity pages on opportunities.creativescotland.com — no authenticated endpoints, recruiter-only features, or content behind a login. Cloudflare WAF rotation is used solely to overcome unintentional IP-reputation false positives that block legitimate scraping; we do not bypass JS challenges, CAPTCHAs, or any security mechanism that requires interactive verification. Users are responsible for ensuring their use complies with opportunities.creativescotland.com's Terms of Service, applicable data-protection law (GDPR, CCPA, etc.), and any contractual obligations of their own organisation.

---

SEO Keywords

creative scotland scraper, scrape opportunities.creativescotland.com, creative scotland opportunities api, scottish creative sector jobs scraper, scottish arts jobs api, scotland workshops scraper, scottish festivals data, scottish creative jobs scraper, Apify creative scotland, cloudflare waf bypass scraper, residential proxy rotation scraper, scotland cultural sector jobs api, scottish gallery jobs scraper, scottish theatre jobs scraper, edinburgh creative jobs scraper, glasgow arts jobs scraper, asva alternative scraper, highlandjobs alternative scraper, s1jobs alternative scraper, scotland creative recruitment data