Calgary Building Permit Status Delta

Durable, informational monitor of the City of Calgary's official "Building Permits" open dataset (Socrata Open Data Portal, data.calgary.ca, dataset c2es-76ed) — free, public, no API key, no account, no captcha, no browser.

This is a mirror-with-diff of a public government dataset, not legal, real-estate, or construction advice. It does not judge whether a project is legal, complete, or approved — it tells you, reliably and cheaply, when a permit's statuscurrent actually changes (Application Received → In Review → Issued Permit → Completed, and more), a new permit shows up in a community's slice of the dataset, or a previously-tracked permit drops out of it (confirmed over two runs).

Why

The city already publishes the full permit table for free (nearly 500,000 rows), but there is no durable "what changed since I last checked" — you either poll the whole table yourself and diff it client-side, or you don't watch at all. Every competing scraper/exporter we found for this dataset filters by application or issue date (i.e. "new applications"); none tracks status transitions of permits already in the pipeline over time. Contractors, real-estate agents, inspectors, and neighbours use a status change (e.g. "Issued Permit", "Completed") as a real, actionable signal — and a permit applied for years ago can still change status today. This actor keeps that diff for you: a list of watches (one Calgary community each), a durable per-permit content hash between runs, and a bill only for permits that are genuinely new, changed, or confirmed gone.

How it works

1. Each watch tracks the entire permit history for one Calgary community (communityCode, the City's own 3-character code, e.g. "DNC" for Downtown Commercial Core) — not a window of recent activity. This is a deliberate, load-bearing difference from a "last N by date" design: a permit applied for months or years ago can change status today, and a date-window monitor would silently miss exactly that transition. Every run therefore re-reads the whole community slice, page by page, sorted deterministically by permitnum, and only accepts it as complete if every page's row count matches the community's declared total (checked fresh on every page) from start to finish.
2. The identity of one tracked item is permitnum alone — confirmed globally unique across the entire 499,681-row dataset (unlike some sibling monitors in this line, no composite key is needed here).
3. The first run for a watch establishes a baseline: every permit in the slice gets a content hash computed from its lifecycle fields (statuscurrent + applied/issued/completed dates) and stored — nothing is billed or delivered. A baseline is only ever committed from a complete slice: any truncated page, any mismatch in the community's declared total between two page reads, an overlapping permitnum across two different pages, or a slice larger than this actor's safety cap leaves the index untouched and reports the run honestly instead of silently producing a partial "everything is new" baseline.
4. Every later run compares the fresh slice against the stored index:
   • a permit never seen before (or previously confirmed removed) is new;
   • a permit seen before whose lifecycle content hash differs is status_changed;
   • a permit present in the index but missing from two consecutive clean full-slice reads is removed — a single missing read (a pagination race on a live table) does not count; the permit simply keeps its history and can return later as a normal status_changed.
5. A hard per-watch cap (maxResultsPerWatch) protects you from a single run billing/delivering an unbounded backlog on the very first post-baseline run — anything over the cap is picked up cleanly on the next run.

Input

{
  "monitorId": "my-calgary-watch",
  "watches": [
    { "watchId": "downtown-core", "communityCode": "DNC" }
  ],
  "maxResultsPerWatch": 200,
  "notifyOn": "new_alerts",
  "webhookUrl": "https://example.com/webhook"
}

• watches — 1-5 objects, each {watchId, communityCode}. communityCode is the City of Calgary's own 3-character community code (case-insensitive, normalized to uppercase) — the City currently maintains 316 of them. A watchId is bound to its communityCode on first use; reusing the same watchId with a different communityCode later fails the run honestly (watch_config_mismatch) instead of silently mixing history under a stale index. A community code that is well-formed but has no permits yet (or a typo that happens to match the 3-character pattern) is not an input error — it simply establishes an honest, empty baseline.
• maxResultsPerWatch — 1-2000, default 200.
• Keep the number of watches small: each run re-scans the whole permit history per community, so large communities (e.g. "DNC", "SAD") take longer and cost more requests than small ones.

Output

One row per new/changed/removed permit: watchId, communityCode, changeType (new/status_changed/ removed), permitNum, status, previousStatus, appliedDate, issuedDate, completedDate, permitType, permitClass, permitClassGroup, permitClassMapped, workClass, workClassGroup, description, applicantName, contractorName, housingUnits, estProjectCost, totalSqft, originalAddress, communityName, contentHash. A run that finds nothing new/changed still writes an honest run_summary row to the default dataset (never silently empty).

Billing

Pay-per-event, one named event:

• permit-status-changed — a permitnum that is new (after baseline), whose status/lifecycle dates changed, or whose two-run-confirmed disappearance from the community's dataset slice we just reported.

Deduplicated by an atomic claim gate keyed on the permit plus a persistent changeSequence (so a cyclical status like Issued Permit → Completed → Issued Permit again — a rare but observed correction in this dataset — bills each real transition once, never zero times and never twice). The baseline run establishes history without charging. A watch that never gets to fetch its slice at all (source fetch incomplete/blocked before any event was claimed, or the run timed out before reaching that watch) is never charged for that watch.

Charging is decided per delivered event, not by the run's final status. A run is only ever marked FAILED as a whole after everything up to the point of failure has already happened for real — so if watch A's events were already delivered and charged earlier in the same run, and watch B then hits a fatal error (e.g. the lease is lost), the run correctly reports FAILED overall, but watch A's already-billed rows are not retracted: they were genuinely delivered, remain visible in the Dataset, and stay charged (reversing a charge that already reflects real, delivered work would itself be dishonest bookkeeping). In other words: FAILED describes the run as a whole, not a promise that nothing in it was ever billed.

Delivery/billing guarantee: at-most-once, not exactly-once

The right to perform an irreversible action (dataset write + PPE charge) is granted by the only atomic primitive Apify offers — RequestQueue.addRequest(uniqueKey) → wasAlreadyPresent — in a separate named queue that acts as a permanent journal (<prefix>-<monitorId>-claims). The Apify Key-Value Store has no CAS, no conditional write and no ETag, so it can only ever be a diagnostic state machine, never the source of at-most-once.

Concretely: for one computed event, delivery and charging each happen no more than once. If the run dies after taking the claim, the event may be lost (it stays dataset_unknown/charge_unknown and is never re-delivered) — but you will never be billed twice. That is a deliberate trade: "never overcharge" beats "never lose a row".

Boundaries of the guarantee, stated honestly:

• Between the internal lease check and the dataset write/charge there is an unavoidable TOCTOU gap; what actually protects your money is the claim gate, not the lease.
• The guarantee holds for as long as the named claims queue exists. Anyone with account access can delete or recreate it via Console/API, which starts the journal from zero. This is a boundary of any durable storage, not a defect of the protocol.

Honest limits

• We mirror the City of Calgary's own published statuscurrent field as-is; we do not judge legality, project completeness, or actual construction progress, and we do not guarantee the city's underlying data is complete or current. removed means the permit left the public dataset slice for that community for two confirmed runs — it is not a statement that the permit was legally closed, cancelled, or approved in any other sense the city does not publish directly in this dataset.
• Unlike a CKAN-based dataset (e.g. our companion Toronto monitor), Socrata's list endpoint does not return a total row count in the page body itself — this actor issues a separate lightweight count(*) query alongside every page read specifically so the same "did the declared total drift between two reads of the same watch" safety check still applies (see the source code for the exact mechanism).
• A baseline (and every later comparison) requires reading the entire history for a watch's communityCode in one run. If the source returns a truncated page, a row-count mismatch, an overlapping permitnum across two pages, or a slice bigger than this actor's safety cap, the run reports that honestly (source_access_limited / watch_capacity_exceeded) and leaves the durable index untouched — never a partial baseline, never a false "everything disappeared".
• permitnum is confirmed globally unique in this dataset (verified against all 499,681 rows); any repeated permitnum within a single page — even with identical content — is treated as a structural anomaly and stops the run for that watch, rather than being silently collapsed.
• If the feed is temporarily unavailable, its shape changes, or a watch's slice can't be confirmed complete, the affected watch reports that honestly instead of silently returning zero results — and any partial run is surfaced in the coverage record, the digest, and the webhook payload, never masked as "no changes found".

Author: OmniCoder (https://t.me/OmniCoder)