UK Planning Applications Lead Finder (Apify actor)

Finds public UK planning applications that look useful for trades, kitchens, extensions and renovation lead generation.

The actor searches supported council planning portals, normalises each result, then adds a relevance score and plain-English reasons such as extension, single-storey extension, kitchen, renovation or loft conversion.

Use it to build a weekly prospecting list for kitchen fitters, builders, architectural services, glazing, roofing, landscaping and other home-improvement businesses.

What you get

Each dataset row includes:

leadId, councilId, councilName, sourceUrl, applicationRef, address, proposal, status, submittedDate, decisionDate, applicantName, relevanceScore, relevanceReasons, matchedKeywords, category, scrapedAt

Typical result:

• council: Bury Council
• proposal: Single storey rear extension
• category: extension
• relevance score: 6
• reasons: matched extension and single-storey extension

Current coverage

Default input uses councilSelection: ["north-west-pack"], focused on Daz's North West lead-gen area. Verified active councils currently include:

• Bury
• Trafford
• Stockport
• Wigan
• Knowsley
• St Helens
• Cheshire West and Chester
• Lancaster
• Blackpool

midlands-pack is also available for Warwick, Solihull, Wolverhampton and Bromsgrove/Redditch.

Supported portal styles

Output schema (dataset records)

leadId, councilId, councilName, sourceUrl, applicationRef, address, proposal, status, submittedDate, decisionDate, applicantName, relevanceScore, relevanceReasons, matchedKeywords, category, scrapedAt

• Dates are normalized to YYYY-MM-DD.
• applicantName is only filled where the portal publishes it on the results page (Idox shows it only on detail pages, so it is null for Idox in this MVP).
• category is one of kitchen, extension, loft, conversion, renovation, new-build, other.
• relevanceScore combines your input keywords (+2 each) with built-in trade signals (kitchen +5, extensions +3–4, refurb/renovation +3, …) and negative signals (tree works, adverts, telecoms −5) so junk can be filtered with minRelevanceScore. relevanceReasons explains every point.

Running on Apify

Use the default North West preset for a quick run, or select specific council IDs in councilSelection. Keep sendKeywordToPortal off unless you deliberately want a narrow exact search; the default date-range search catches more relevant applications and filters locally.

Recommended starter input:

{
  "councilSelection": ["north-west-pack"],
  "keywords": ["kitchen", "extension", "refurbishment", "alterations"],
  "minRelevanceScore": 1,
  "maxPagesPerCouncil": 2,
  "maxConcurrency": 1,
  "requestDelaySecs": 5,
  "sendKeywordToPortal": false
}

Legacy inputs such as councils: ["bury"] and

councilPreset: "north-west-pack"

Local run

npm install
npm test              # offline — fixtures only, no network
npm run check:councils # live compatibility matrix, writes AVAILABILITY.md
apify run             # local Apify-style run using src/sampleInput.json
npm start             # same actor entrypoint without the CLI wrapper
npm run smoke         # optional: one polite live request to the first active sample council

Results land in ./storage/datasets/default/*.json. To run with your own input locally, create ./storage/key_value_stores/default/INPUT.json matching .actor/input_schema.json, or edit src/sampleInput.json.

Politeness / rate limiting

Defaults are deliberately conservative and configurable via input: maxConcurrency: 1, requestDelaySecs: 5 between requests to the same council, maxPagesPerCouncil: 3, 1 retry, and an identifying user agent. Please keep them low — council portals are small public services.

Publishing / monetisation notes

The actor is designed to run as a small paid utility on Apify. Good first-public positioning:

• title: UK Planning Applications Lead Finder
• audience: trades, kitchen firms, builders, renovators and local lead-gen teams
• promise: Find recent planning applications likely to indicate upcoming home-improvement work
• pricing: start with Apify usage-based pricing while coverage is still growing
• caveat: it is not a marketing-compliance tool; users remain responsible for lawful basis, council terms, and opt-outs

Legal & privacy caution

• Planning registers are public, but check each council's terms of use and robots.txt before scraping, and keep request rates minimal.
• Records contain personal data (addresses, sometimes applicant names). Under UK GDPR you need a lawful basis (likely legitimate interests) to process them for marketing; run and document an LIA before contacting anyone.
• Direct mail to addresses from the register is generally lower-risk than calls/emails; screen against mail preference services and honour opt-outs.
• Do not resell raw register data; some councils assert database rights.
• Applicant/agent details published on portals must not be used contrary to the council's stated purpose notices.

Pilot councils & next ones to add

Sample input uses councilSelection: ["north-west-pack"], which expands to every active North West council in src/councils.js. Current verified active North West councils are Bury, Trafford, Stockport, Wigan, Knowsley, St Helens, Cheshire West and Chester, Lancaster, and Blackpool; midlands-pack remains available for Warwick/Solihull/Wolverhampton/Bromsgrove-Redditch. npm run check:councils writes the latest live matrix to AVAILABILITY.md. Council portal URLs change often — verify the base URL loads before relying on it.

Good next candidates (all Idox PublicAccess at the time of writing):

• South Kesteven / Boston / East Lindsey (Lincolnshire cluster)
• Maidstone, Medway, Canterbury (Kent)
• Trafford, Stockport, Bolton (Greater Manchester)
• Councils on Northgate/Planning Explorer or custom portals need either a new portalType parser or per-council aspxParams mapping — the header-matching table parser (parseAspxSearchResults) is the starting point.

To add a council: append { name, portalType, baseUrl } to the input (or src/sampleInput.json), run npm run smoke-style single fetch to confirm the markup parses, and add a fixture + test if the markup differs.

Project layout

.actor/            actor.json + input_schema.json (Apify metadata)
src/main.js        entrypoint: input → crawl → score → dataset
src/scraper.js     URL builders + HTML parsers (Idox, ASPX table)
src/leadScoring.js keyword/relevance scoring + categorisation
src/sampleInput.json  fallback input for local dev
test/              offline tests + HTML fixtures (node --test)
scripts/smoke.js   optional single live fetch, non-fatal offline