# Changelog of Licensed Contractor Leads Scraper — CA, FL, TX, NY (`sleek_waveform/construction-contractor-leads`) Actor

- **URL**: https://apify.com/sleek_waveform/construction-contractor-leads/changelog.md
- **Full Actor documentation**: https://apify.com/sleek_waveform/construction-contractor-leads.md

## Changelog

### \[1.1.5] - 2026-08-11

#### Fixed — run killed as TIMED_OUT before producing anything

- **Stream the CSLB master CSV instead of buffering it.** `getMasterFilePath()`
  downloaded the whole ~74 MB export (`await resp.arrayBuffer()`) to `/tmp` before
  a single row could be parsed. CSLB serves that file at ~0.1-0.5 MB/s, so the
  first record only existed minutes in — and a run with a short timeout was killed
  as TIMED_OUT having produced nothing, even though matching contractors sit on the
  first data row. Rows are now parsed as bytes arrive and the transfer is aborted
  as soon as `maxResults` matches are found.
  Measured (`state=CA, activeOnly, licenseType=all`): **maxResults=2 → 2.4s** and
  **maxResults=50 → 2.5s**, versus 125-206s for the same cohort before. Most runs
  now read a few MB instead of 74 MB.
- **The `/tmp` cache is gone.** Each Apify run gets a fresh container, so it never
  hit on the platform — it only forced the full download.
- **Work against a run deadline.** Derived from the platform's `timeoutAt` less a
  15% margin clamped to \[10s, 25s]: the CSV scan, the CSLB HTTP range scan and the
  Socrata paging loop all stop starting work past it, and a watchdog finalises and
  exits 0 if something stalls inside a single await. A run that runs out of time
  now finishes SUCCEEDED with the records it collected instead of TIMED_OUT with
  none. Per-request socket timeouts are capped by the remaining budget, so one hung
  request can no longer outlive the run.
- **Dropped `environmentVariables` from `.actor/actor.json`.** It pinned a dead
  `APIFY_MONITORING_URL` (`openclaw-matrix.duckdns.org`) that `apify push` bakes
  over the correct platform value — the last owner run log shows the actor
  registering its failure webhook against that dead host. Removing the key makes
  the platform value authoritative and survive pushes.

#### Tests

- `src/__tests__/cslbBulk.stream.test.ts` — streaming contract: yields from a
  transfer that never finishes, releases the socket at `maxResults` and on early
  break, honours the deadline, and surfaces a truncated transfer instead of
  reporting it as a complete file. Each was mutation-checked (restoring the
  buffering, removing the deadline check, removing the close) to confirm it fails
  with the bug present.
- `src/__tests__/run-scheduling.test.ts` — deadline margin/derivation and
  request-timeout clamping.

### \[1.1] - 2026-07-12

#### Performance & revenue (apify-improve, Pattern E)

- **Default CA searches use CSLB Master CSV** — top cohort (`maxResults=50`, no filters) was averaging ~28 charged records via random HTTP probing; bulk stream delivers exact counts with phone/address.
- **Bulk → HTTP → Playwright fallback chain** when CSV download fails.
- **Batched pushData + charge** (25 records) on bulk path.
- **`src/run-scheduling.ts`** + regression tests; `.actor/test-ca-default.json` cohort input.

### \[1.0] - 2026-03-12

#### Added

- Initial release
- Search public state contractor licensing boards (CA, FL, TX, NY) for licensed contractor data
- Filter by license type, city, and active status
- Extract contractor name, business name, license details, and contact information from public records