# Building Permits Scraper – Roofing, Solar, Pool & Remodel Leads (`staticxviper/permits`) Actor

Recent building permits from verified city open-data portals for contractor and exterior-cleaning leads.

- **URL**: https://apify.com/staticxviper/permits.md
- **Developed by:** [CJ Moulton](https://apify.com/staticxviper) (community)
- **Categories:** Lead generation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $5.00 / 1,000 permit rows

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Building Permit Leads

Recent building permits from verified city and county open-data portals. Issue #128. Aimed at contractor and exterior-cleaning leads: roof, siding, pool, deck, solar, and new construction.

The Actor reads `sources.json` in this folder and reuses `scripts/lead_automation/new_business_sources/open_data.py` and `http_table.py` for Socrata page URLs and HTML tables. Those modules stay the source of truth for polite HTTP and query strings. Permit field mapping lives in this Actor because the shared adapters emit business-formation records.

Low-volume research. Each feed is requested with a polite delay. The default pause is 2 seconds and jitters upward by up to 2 seconds, which covers the Crawl-delay of 1 on the SODA robots files. There is no browser and no stealth plugin.

Coverage is narrow on purpose. A portal is queried only after its open-data feed is verified in `sources.json`.

### What it does

Checked 2026-10-26 with a one-row HTTPS read of each resource:

| City | Source id | Endpoint |
|------|-----------|----------|
| Chicago | `chicago_building_permits` | `https://data.cityofchicago.org/resource/ydr8-5enu.json` |
| Seattle | `seattle_building_permits` | `https://data.seattle.gov/resource/76t5-zqzr.json` |
| Austin | `austin_issued_permits` | `https://data.austintexas.gov/resource/3syk-w9eu.json` |
| Los Angeles | `los_angeles_building_permits` | `https://data.lacity.org/resource/pi9x-tg5x.json` |

Rows are filtered by `dateFrom` / `dateTo`, optional permit types, and optional `minValue`, then deduped on `(source, permit_id)`. The same permit id from two cities stays two rows.

`owner_name` is filled only when that portal publishes an owner. Chicago copies a contact when `contact_N_type` contains "owner", and a contractor when it contains "contractor". Seattle and Austin publish a contractor company and no owner column, so `owner_name` is empty. Los Angeles publishes the site address, value, and work description, and does not publish a contractor or an owner. Chicago does not publish a property ZIP, so `zip` is empty on that feed.

An `http_table` source can be added when a city publishes a plain HTML table and that page has been checked the same way. Query style `arcgis` is supported for a FeatureServer that has been verified the same way. The current catalog is Socrata.

Access ladder, cheapest path first:

1. Direct HTTP.
2. Apify Proxy only after a real block (HTTP 401, 403, 429, or a challenge page). The group comes from the input. An empty group uses datacenter. Residential is used only when the input selects it.
3. A CAPTCHA solve only when that proxy retry is still a challenge, `allowCaptcha` is true, and `maxSpendUsd` covers the solve.

A robots disallow is not retried through a proxy. If the run is still blocked and the proxy budget is gone, or CAPTCHA is off, or no solver is configured, the Actor pushes one `blocked` row and stops. It exits 0. This package does not ship a third-party solver key, so `allowCaptcha: true` still stops unpaid until a solver is configured.

### Input

`fixtures/INPUT.json`:

```json
{
  "sources": ["chicago_building_permits"],
  "dateFrom": "2026-09-26",
  "dateTo": "2026-10-26",
  "permitTypes": ["roof", "siding", "pool", "deck", "solar"],
  "minValue": 1000,
  "maxItems": 25,
  "maxRequests": 24,
  "delaySeconds": 2,
  "maxSpendUsd": 5,
  "maxProxyCostUsd": 0.5,
  "allowCaptcha": false,
  "headless": true,
  "proxyConfiguration": {"useApifyProxy": false}
}
```

Leave `sources` empty to query every verified feed. `dateFrom` defaults to 30 days before the run. `dateTo` defaults to the run date. `permitTypes` accepts `roof`, `siding`, `pool`, `deck`, `solar`, and `new construction`. Roof also matches reroof and shingle. `minValue` of 0 keeps blank job values. A minimum above 0 drops blanks. `maxItems` defaults to 50 and cannot exceed 200. `maxSpendUsd` defaults to 5 and cannot exceed 100. `maxProxyCostUsd` defaults to 0.50 and is clamped to the spend cap. `allowCaptcha` defaults to false. `headless` is kept for the shared Actor contract and has no effect.

### Output

One dataset item per permit, plus a status row when a source is blocked, disallowed, empty, or the input is wrong. Key-value `SUMMARY` records found, billedPermits, proxyCharges, captchaCharges, mergedDuplicates, spendUsd, and one object per source (`id`, `status`, `fetched`, `kept`). The run log writes that same per-source line.

```json
{
  "permit_id": "B-EXAMPLE-1",
  "issued_date": "2026-10-12",
  "type": "PERMIT - RENOVATION - Roof",
  "description": "Tear off and re-roof with architectural shingles",
  "value_usd": 18500,
  "address": "100 N EXAMPLE AVE",
  "city": "Chicago",
  "state": "IL",
  "zip": "",
  "lat": 41.5,
  "lng": -87.5,
  "contractor_name": "Example Roofing Co",
  "contractor_license": "",
  "owner_name": "Example Owner",
  "source": "chicago_building_permits",
  "source_url": "https://data.cityofchicago.org/resource/ydr8-5enu.json?permit_=B-EXAMPLE-1",
  "sourceStatus": "found",
  "ok": true,
  "error": "",
  "scrapedAt": "2026-10-26T21:00:00+00:00"
}
```

| sourceStatus | Meaning | permit event |
|--------------|---------|--------------|
| `found` | A permit was returned | yes, while `maxSpendUsd` still covers it |
| `not_found` | The feed answered and nothing matched | no |
| `blocked` | Direct request was rejected and the run could not continue inside the caps | no |
| `robots_disallowed` | robots.txt disallows the feed | no |
| `error` | The fetch failed, the request cap was hit, or the input was incomplete | no |

### Limitations

- Verified feeds today are Chicago, Seattle, Austin, and Los Angeles. Schemas differ. A city is not added until a live one-row read confirms the columns.
- The NYC DOB permit-issuance resource `ipu4-2q9a` had no rows with `issuance_date` after 2025-01-01 on 2026-10-26, so it is not in the catalog. San Francisco resource `i98e-djpj` returned 404. Los Angeles resource `nbyu-2ha9` returned 403; `pi9x-tg5x` is the one this Actor calls.
- Chicago job value is `reported_cost`. Austin is `total_job_valuation`. Los Angeles is `valuation`. Seattle is `estprojectcost`. A blank value is kept only when `minValue` is 0.
- `maxItems` applies after the date, type, and value filters and after dedupe. `truncated` in `SUMMARY` is true when that cap stopped the run. Permit charges never exceed `maxItems`.
- Once `maxSpendUsd` cannot cover the next permit event, later permits are saved without a success charge.
- Empty, failed, blocked, and robots runs do not emit `permit`.

### Robots and terms

robots.txt is fetched for each host and evaluated with `scripts/lead_automation/crm_enrich_robots.py` (longest match, `User-agent: *`). The verified SODA feeds allow `/resource/` and publish Crawl-delay 1. A shorter `Disallow` on browse and catalog URLs does not hide `/resource/`. A disallowed feed is not fetched and is not retried through a proxy.

Terms noted when the feeds were verified (2026-10-26):

- Chicago, Seattle, Austin, and Los Angeles publish these extracts on their Socrata open-data portals. The rows are public data for research. They are not certified permit records and they are not a substitute for the city's permit file.

### Pricing

Pay per event. Prices are the starting Store rates in `.actor/pay_per_event.json`.

| Event | Price | When it is charged |
|-------|-------|--------------------|
| `permit` (primary) | $0.05 | A permit row is pushed |
| `proxy` | $0.01 | One request was sent through Apify Proxy after a direct block |
| `captcha` | $1.00 | A solver returned a token after the proxy retry was still a challenge |

Direct requests are free. A challenge with `allowCaptcha` false, or with no solver configured, stores a `blocked` row and does not emit `captcha`. A run that pushes no permits does not emit `permit`. Dataset items are not a charge event. The platform still applies its start event. Memory default is 1024 MB and the default timeout is 900 seconds.

Proxy uses the input group, or datacenter when the input leaves the group empty, and only after a block. The proxy password is the platform `APIFY_PROXY_PASSWORD`. It is not logged.

### Categories

Set in the Apify Console, not in `actor.json`. Suggested: Business, Real estate.

### Local run

```bash
cd scripts/apify-scripts/permits
mkdir -p storage/key_value_stores/default
cp fixtures/INPUT.json storage/key_value_stores/default/INPUT.json
apify run --purge
```

Unit tests read fixtures only:

```bash
python -m unittest unittests.apify_scripts.test_permits
```

Publish with `APIFY_TOKEN` (`apify push`). That token is not `APIFY_API_KEY`. See `scripts/apify-scripts/README.md`.

# Changelog

This Actor's version history is a separate document: https://apify.com/staticxviper/permits/changelog.md

# Actor input Schema

## `sources` (type: `array`):

Preset source ids. Leave empty to query every verified feed. Chicago, Seattle, Austin, and Los Angeles were checked on 2026-10-26.

## `dateFrom` (type: `string`):

Keep permits issued on or after this date (YYYY-MM-DD). Default is 30 days before the run.

## `dateTo` (type: `string`):

Keep permits issued on or before this date (YYYY-MM-DD). Default is the run date.

## `permitTypes` (type: `array`):

Optional. Matched against the permit type and description. Roof also matches reroof and shingle. Leave empty for every type.

## `minValue` (type: `number`):

Drop permits below this job value. Default 0 keeps every value, including rows the city left blank. A minimum above 0 drops blank values.

## `maxItems` (type: `integer`):

Cap on permit rows pushed for this run, after dedupe. Default 50, maximum 200. Status rows do not count toward this cap and are not billed.

## `maxRequests` (type: `integer`):

Cap on requests, including robots.txt and a proxy retry. Default 24, maximum 80.

## `delaySeconds` (type: `number`):

Seconds to wait before each request after the first. Default 2, which covers the Crawl-delay of 1 on the SODA robots files. The pause jitters upward by up to 2 seconds.

## `maxSpendUsd` (type: `number`):

Stop charging when the next permit, proxy, or CAPTCHA event would pass this amount. Default 5, maximum 100. Later permit rows are saved and are not billed.

## `maxProxyCostUsd` (type: `number`):

Cap on proxy events alone. Default 0.50. A direct request is always tried first. Proxy is used only after a block, and only when this cap and maxSpendUsd both still have room. 0 skips the proxy retry.

## `allowCaptcha` (type: `boolean`):

Default false. When false, a challenge after the proxy retry pushes a blocked row and exits without a solve charge. When true, a solve is attempted only if the proxy retry is still a challenge and maxSpendUsd covers the captcha event. This Actor does not ship a paid solver, so an unconfigured solve pushes blocked and is not billed.

## `headless` (type: `boolean`):

Kept for the shared Actor contract. This Actor does not launch a browser.

## `proxyConfiguration` (type: `object`):

Apify Proxy group used only after a direct request is blocked. Leave Apify Proxy off so the first attempt stays direct. Empty groups use datacenter. Residential is used only when this field selects it.

## `proxyGroups` (type: `array`):

Optional group names, such as DATACENTER or RESIDENTIAL. Used after a block when proxyConfiguration has no apifyProxyGroups. The first request stays direct.

## Actor input object example

```json
{
  "dateFrom": "2026-09-26",
  "dateTo": "2026-10-26",
  "minValue": 0,
  "maxItems": 50,
  "maxRequests": 24,
  "delaySeconds": 2,
  "maxSpendUsd": 5,
  "maxProxyCostUsd": 0.5,
  "allowCaptcha": false,
  "headless": true,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `results` (type: `string`):

Dataset containing all scraped items

## `files` (type: `string`):

Key-value store containing the SUMMARY record for this run

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "dateFrom": "2026-09-26",
    "dateTo": "2026-10-26"
};

// Run the Actor and wait for it to finish
const run = await client.actor("staticxviper/permits").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "dateFrom": "2026-09-26",
    "dateTo": "2026-10-26",
}

# Run the Actor and wait for it to finish
run = client.actor("staticxviper/permits").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "dateFrom": "2026-09-26",
  "dateTo": "2026-10-26"
}' |
apify call staticxviper/permits --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,staticxviper/permits"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/C4vijjaP2gSQLhO6Y/builds/aZAb0XjxgtgCD6AAo/openapi.json