Permit Signal Scout

Permit Signal Scout turns selected official public building permit datasets into normalized, ranked construction opportunity signals.

It reads public Socrata open-data endpoints, normalizes source-specific permit records into one schema, scores each record with deterministic rules, and reports source health so users can see whether the data feed behaved as expected.

What It Does

• Fetches recent public construction permit records from selected official city open-data sources.
• Normalizes permit IDs, issue dates, permit types, descriptions, addresses, contractor/applicant/company names when available, estimated values, and statuses.
• Classifies obvious trade signals such as roofing, HVAC, solar, plumbing, electrical, remodeling, ADU, multifamily, commercial, restoration, and general construction.
• Scores records by freshness, estimated value, trade fit, and completeness.
• Returns ranked permit opportunity signals with score reasons and risk flags.
• Produces a run summary with per-source health.

Who It Is For

• Contractors watching for nearby project activity.
• Construction suppliers monitoring demand signals.
• Real estate investors tracking permit activity.
• Agencies building research-backed prospect lists manually.
• Data buyers who need normalized permit feeds from selected cities.

Supported Sources / Cities

city id	city	source	dataset
austin_tx	Austin, TX	Austin Open Data	3syk-w9eu
chicago_il	Chicago, IL	Chicago Data Portal	ydr8-5enu
nyc_ny	New York City, NY	NYC Open Data DOB NOW	rbx6-tga4
cincinnati_oh	Cincinnati, OH	Cincinnati Open Data	uhjb-xac9
san_francisco_ca	San Francisco, CA	DataSF	i98e-djp9

What It Does Not Do

• It does not find emails.
• It does not verify owners or decision makers.
• It does not guarantee contractor identity.
• It does not send outreach.
• It does not create ready-to-email lists.
• It does not provide legal, compliance, or business advice.
• It does not currently cover every city.
• It does not use AI scoring in the MVP.
• It does not perform contact enrichment, skip tracing, CRM sync, code-violation enrichment, or paid API calls.

Permit Signal Scout provides permit opportunity signals, not verified sales leads.

Example Use Cases

• Find recent high-value roofing permits across the supported cities.
• Monitor HVAC/mechanical permit activity in Austin and Chicago.
• Compare recent permit volume and source health across all supported sources.
• Pull raw normalized records for debugging a city adapter.
• Build an internal research workflow around public permit activity.

Input Fields

field	type	default	description
cities	string array	all supported cities	Source city IDs to query.
issuedAfter	ISO date string	today minus 30 days	Include permits issued on or after this date.
issuedBefore	ISO date string or null	null	Optional upper issued-date bound.
tradeKeywords	string array	default trade terms	User keywords considered during trade matching.
minEstimatedValue	number or null	null	Optional minimum estimated permit value. Records with null values are excluded when this is set.
maxResultsPerCity	integer	500	Maximum records fetched per city, 1-5000.
includeAiScoring	boolean	false	Reserved for later. MVP uses deterministic scoring and emits a warning if true.
outputMode	string	ranked	Accepted values: ranked, byCity, rawNormalized.
includeRawRecord	boolean	false	Include original Socrata row on each output record.
sortBy	string	leadQualityScore	Accepted values: leadQualityScore, issuedDate, estimatedValue, sourceCity.
leadGradeFilter	string array	["A","B","C","D"]	Return only selected lead grades after scoring.

Example Input

{
  "cities": ["austin_tx", "chicago_il", "nyc_ny"],
  "issuedAfter": "2026-05-01",
  "issuedBefore": null,
  "tradeKeywords": ["roof", "roofing", "reroof", "membrane"],
  "minEstimatedValue": 25000,
  "maxResultsPerCity": 250,
  "includeAiScoring": false,
  "outputMode": "ranked",
  "includeRawRecord": false,
  "sortBy": "leadQualityScore",
  "leadGradeFilter": ["A", "B"]
}

More examples are in examples/.

Output Fields

Each output record follows this normalized shape:

field	description
permitId	Permit identifier from the source, or blank if missing.
sourceCity	Source city ID.
sourceSystem	Always Socrata for the MVP.
sourceDataset	Socrata dataset ID.
sourceUrl	Source record link when available or safely generated, otherwise null.
issuedDate	Normalized ISO date or null.
permitType	Normalized permit/work type text.
description	Source work description.
address	Normalized address text when available.
contractorName	Contractor/applicant/company field when available and appropriate.
estimatedValue	Parsed estimated value or null.
status	Source permit status.
matchedTrades	Matched trade categories or user keyword matches.
freshnessScore	0-100 issued-date freshness score.
valueScore	0-100 estimated-value score.
tradeFitScore	0-100 trade relevance score.
completenessScore	0-100 record completeness score.
leadQualityScore	Weighted opportunity score rounded to one decimal.
leadGrade	A, B, C, or D.
scoreReasons	Concise explanation of why the record scored as it did.
riskFlags	Missing/ambiguous/stale-data flags.
rawRecord	Optional original source row when includeRawRecord is true.

Example Output

{
  "permitId": "SAMPLE-2026-001",
  "sourceCity": "austin_tx",
  "sourceSystem": "Socrata",
  "sourceDataset": "3syk-w9eu",
  "sourceUrl": "https://example.test/public-permit/SAMPLE-2026-001",
  "issuedDate": "2026-05-20",
  "permitType": "Building Permit / Roofing",
  "description": "Roof replacement and related exterior repair",
  "address": "123 SAMPLE ST",
  "contractorName": "SAMPLE ROOFING LLC",
  "estimatedValue": 45000.0,
  "status": "Issued",
  "matchedTrades": ["roofing"],
  "freshnessScore": 100,
  "valueScore": 75,
  "tradeFitScore": 100,
  "completenessScore": 100,
  "leadQualityScore": 93.8,
  "leadGrade": "A",
  "scoreReasons": [
    "recent permit",
    "solid estimated value",
    "matched roofing trade",
    "complete record",
    "contractor/applicant present"
  ],
  "riskFlags": []
}

This is a synthetic example. It shows the output shape, not a verified sales lead.

Run Summary

The run summary reports:

• totalRecordsFetched
• totalRecordsReturned
• recordsByCity
• averageLeadQualityScore
• topMatchedTrades
• sourceHealth
• warnings

examples/run_summary_sample.json shows the expected shape.

Scoring Model

The MVP uses deterministic scoring only:

leadQualityScore =
  freshnessScore * 0.30 +
  valueScore * 0.25 +
  tradeFitScore * 0.25 +
  completenessScore * 0.20

leadQualityScore is rounded to one decimal.

Freshness score:

• 0-7 days old: 100
• 8-30 days old: 85
• 31-90 days old: 65
• 91-180 days old: 40
• 181+ days old: 15
• missing date: 0
• future date: 50 and flagged as stale_record

Value score:

• null: 35
• under $5,000: 25
• $5,000-$24,999.99: 50
• $25,000-$99,999.99: 75
• $100,000-$500,000: 90
• over $500,000: 100

Trade fit score:

• exact category keyword in permit type: 100
• exact category keyword in description: 85
• related keyword signal in description: 65
• generic construction/remodel signal: 45
• no match: 10

Completeness score:

• +20 permit ID
• +20 issued date
• +20 address
• +15 description
• +15 permit type or status
• +10 contractor/applicant/company

Lead Grades

grade	score range
A	80-100
B	65-79.999
C	45-64.999
D	0-44.999

Grades are opportunity-signal grades, not buyer-intent guarantees.

Source Health

Each selected source reports:

• status: healthy, empty, degraded, or failed
• recordsFetched
• recordsReturned
• latestIssuedDate
• warnings
• errors

Status meanings:

• healthy: fetch and pipeline succeeded, or the source produced a valid result.
• empty: fetch succeeded but no records matched.
• degraded: partial failure, malformed rows skipped, or warnings crossed a threshold.
• failed: source fetch failed completely or all fetched rows failed normalization.

Warnings use stable codes such as:

• source_returned_zero_rows
• source_fetch_failed
• records_skipped_missing_date
• records_skipped_min_estimated_value
• malformed_records_skipped
• includeAiScoring_requested_disabled_mvp
• source_latest_date_stale
• maxResultsPerCity_capped_results

Limitations

• Coverage is limited to five supported public datasets.
• Public data can lag, change schema, or include incomplete fields.
• Some sources do not provide contractor fields.
• Some contacts may be applicants, owners, or permit representatives, not sales contacts.
• Estimated values may be missing, rounded, entered inconsistently, or source-specific.
• Trade matching is deterministic keyword matching, not legal or professional classification.
• Source URLs are not available for every dataset.

Ethical / Public-Data Boundaries

Use this Actor for lawful public-data analysis, market research, and operational planning.

Do not use it for spam, harassment, deceptive outreach, unlawful profiling, or automated contact enrichment. If you use public permit data for sales research, you are responsible for compliance with applicable laws, platform rules, and local norms.

Troubleshooting

issue	likely cause	what to check
No records returned	Date/value/grade filters too narrow	Lower minEstimatedValue, widen dates, include more grades
Source is empty	Fetch worked but filters matched nothing	Check issuedAfter, issuedBefore, and source health
Source is degraded	Partial failure or malformed rows	Inspect sourceHealth[city].warnings
Source is failed	Endpoint failure or all rows failed normalization	Inspect sourceHealth[city].errors
includeAiScoring warning	AI scoring is disabled in MVP	Leave includeAiScoring as false
Missing contractor fields	Source may not provide reliable contractor data	Treat output as permit intelligence, not verified leads

Local Development / Test Commands

Run the unit test suite:

$.venv/bin/pytest -q -s actors/permit-signal-scout/test/test_schema.py actors/permit-signal-scout/test/test_socrata_client.py actors/permit-signal-scout/test/test_source_adapters.py actors/permit-signal-scout/test/test_filters_dedupe.py actors/permit-signal-scout/test/test_scoring.py actors/permit-signal-scout/test/test_main_pipeline.py actors/permit-signal-scout/test/test_source_health.py actors/permit-signal-scout/test/test_examples.py actors/permit-signal-scout/test/test_local_actor_packaging.py

Run a tiny read-only endpoint smoke test:

$.venv/bin/python actors/permit-signal-scout/scripts/smoke_fetch_limit1.py

Run a small read-only local sample:

$.venv/bin/python actors/permit-signal-scout/scripts/run_local_sample.py

Run launch readiness:

$.venv/bin/python actors/permit-signal-scout/scripts/launch_readiness_check.py

Run local Actor-style storage output without publishing:

$.venv/bin/python actors/permit-signal-scout/scripts/run_actor_local.py --input actors/permit-signal-scout/examples/input_local_smoke.json --storage-dir actors/permit-signal-scout/.local-storage

This writes records to datasets/default/*.json and the run summary to key_value_stores/default/RUN_SUMMARY.json under the selected storage directory.

Launch Readiness

The launch readiness check fetches a small sample from the five supported public Socrata sources, runs the full local pipeline, prints source health, prints the top five records, and returns a verdict:

• PASS: all selected sources are healthy or empty with no critical errors.
• WARN: one or more sources degraded/failed but the pipeline still returned records.
• FAIL: all sources failed or no useful pipeline result could be produced.

Next Step

PS-011 adds local Apify-compatible packaging. The next step is a private Apify test run after explicitly confirming the packaging and local storage output.