# Changelog of CA Labor Commissioner Judgments, Wage Claims, Port Drayage List (`overlookdata/ca-dlse-judgments`) Actor - **URL**: https://apify.com/overlookdata/ca-dlse-judgments/changelog.md - **Full Actor documentation**: https://apify.com/overlookdata/ca-dlse-judgments.md ## Changelog All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ### \[0.4.0] - 2026-09-30 #### Added - README "Personal data" section now carries the FCRA framing: the dataset is not a consumer report, the publisher is not a consumer reporting agency, and no eligibility use is permitted; the actor republishes state records without enrichment or edits, and corrections flow through from the source. Citation checked against the live text of 15 U.S.C. 1681a. - docs/STORE-LISTING.md: the Apify Store listing package as published on 2026-09-30 (publication fields with character counts, PPE pricing with the per-result price scaled to the corpus, Console defaults, sample dataset, personal-data posture). - docs/assets/icon-golden-seal.png and .svg, the family icon shared with the other CA DIR actors. - snapshots/ is git-ignored; it holds local full-sweep archives such as the 675,015-record acceptance baseline. #### Changed - Project status files record the 0.3.0 acceptance run (675,015 records, 46 minutes, $3.54 measured cost, peak memory 566MB of 1024MB) and the Store publication (PPE result price $0.00003, Console timeout 7200s). ### \[0.3.0] - 2026-09-29 #### Fixed - Full runs no longer hold the whole corpus in memory. Records mode streams each slice and keeps only per-slice labelled counts and dedupe keys, so the \~670k-record corpus runs in 1024 MB instead of being OOM-killed at 512 MB. - A resumed run (resurrection or platform migration) no longer re-pushes records. The 0.1.1 build re-enumerated the source on resume and wrote the corpus roughly twice; verification now reconciles checkpointed slices by their stored counts and never merges inside a checkpointed window. - The checkpoint is saved when each primary sweep finishes, on the platform's MIGRATING and PERSIST\_STATE events, and every 50 slices, under a lock with detached snapshots so saves land in order. Once a migration begins the run stops pushing, pushes only the verification merges its checkpoint covers (recording their counts first), and reboots to resume instead of exiting. - The port drayage fetch honors the checkpoint, so a resumed run no longer duplicates its rows. - An empty judgment status picklist now fails verification loudly instead of vacuously reporting a match. - Colorado workbook tabs are validated per sheet (claim, employer, amount and address columns) and skipped with a warning naming what is missing; a judgment case with disagreeing amount rows now deterministically keeps the largest amount. #### Changed - Default actor memory is 1024 MB. Entities mode still holds the corpus and needs 4096 MB, now documented in the input schema and README, along with run-timeout guidance for full runs. - Checkpoints use schema version 2 (labelled per-slice counts, fully validated); an old or malformed checkpoint logs a warning and starts fresh. - The CLI resume file is written every 50 slices and on exit, matching the actor's checkpoint cadence. ### \[0.2.0] - 2026-09-29 #### Changed - Project status files record the 0.1.0 release, the Apify push (build 0.1.1, actor mCxAWNmBS0krvFSgv), and the first full platform run: 670,559 unique records measured (wage claims 629,776 back to 1997, judgments 31,027, all six spine feeds match=True, zero slice errors). - TODO.md carries the 0.3.0 platform-fix plan the full run exposed: stream records instead of retaining the corpus in memory (512MB OOM), resume-aware verification (a resumed run re-pushed ~660k duplicate rows through the union-merge path), a larger default memory plus run-timeout guidance, and done\_slices for fetch\_port\_drayage. ### \[0.1.0] - 2026-09-29 #### Added - Aura client for the CA DIR Lightning sites: form builder for the unauthenticated `aura.ApexAction.execute` endpoint, framework-config harvester that decodes the URL-encoded `fwuid` and loaded hash out of a site landing page, self-healing from the framework id every response reports, one re-harvest and retry on an unusable body, and a row-cap detector. - California judgment sweep with per-judgment dollar amounts: entry-date year slices from 2000, a catch-all slice for everything before the corpus starts, and a far-future slice that catches DLSE's data-entry typos in 2029 and 2205. A row is one liable party, so a judgment naming three defendants keeps all three, keyed on the judgment id plus the normalized employer name. The state's own `judgmentPartyId` is not stable across queries or runs, so it is published as an informational field only. - Port drayage unsatisfied-judgment list (Labor Code 2810.4) in one request, plus an `asOfDate` back-fill that replays one snapshot per month and reconstructs the whole add-and-remove history. - Wage-claim sweep, one request per docket day from 2010 plus one fixed catch-all slice for the earlier corpus, which reaches back to 1997 and holds about 31,000 records the catch-all collects in roughly 37 requests. Defaults to defendant-side rows only, so the output does not carry the workers who filed the claims. Narrowing the first docket date narrows the run: the catch-all is a constant, never derived from the input, so it can never widen into the corpus. - Free multi-state spine: WA debarred contractors (form POST download; a plain GET returns HTTP 500), NJ WALL spreadsheet, CO wage theft transparency sheet read from the xlsx export across all nine tabs (the CSV export serves only the active tab), IL debarred contractors parsed from free text, OR labor contractor licensees found by the spreadsheet link the page publishes each month, and Cal/OSHA penalties over $100,000. - Self-verification per dataset: judgments re-enumerated by judgment status over exactly the same date windows as the primary sweep, wage claims recounted by calendar month with an explicit zero-date-gap assertion, the port drayage history checked for missing months, and every feed checked for a parsed row count with its `Content-Length` and `Last-Modified`. Records only the second pass found are merged into the output. Records a status pass structurally cannot reach (no status at all, or a status DLSE omits from its own picklist, such as `Pending/Open`) are counted apart rather than reported as a coverage miss. The whole comparison is written to the `VERIFICATION` key-value record, including when the picklist request itself fails. - Cap handling: a response of exactly 5,000 rows is treated as the cap, the slice is halved and re-fetched down to a single day, and a single day still at the cap is recorded as an error instead of being silently truncated. - Two output modes: `records` (one flat row per source record, identical fields across all ten sources) and `entities` (one row per employer keyed on a hash of the normalized name and address, with total judgment amount, open judgment count, and port drayage, WA debarment and NJ WALL flags). - Checkpoint and resume: completed slice keys are written to the key-value store every 50 slices and on Apify's migration event, and a run resumes from them when the input fingerprint still matches. A slice that failed is never checkpointed as complete, and rows that no slice callback carried (feeds, partially covered slices, verification merges) are emitted once at the end from an explicit list rather than by position. - Apify actor wrapper (`src/main.py`), local CLI runner (`src/cli.py`), input, dataset and output schemas, and an offline pytest suite that runs entirely against saved fixtures through `httpx.MockTransport`.