# Changelog of Career Site Jobs Feed: New Job Postings Monitor (`fecklesspleb/career-site-jobs-feed`) Actor - **URL**: https://apify.com/fecklesspleb/career-site-jobs-feed/changelog.md - **Full Actor documentation**: https://apify.com/fecklesspleb/career-site-jobs-feed.md ## Changelog ### Unreleased: one free status row when every board fails or is blocked (GAP17a) - **Behaviour change (exit code 1 -> 0):** when every entry fails or is blocked (network or HTTP errors, no board found, robots.txt or rate-limited hosts, or private addresses on the Apify platform), the run no longer FAILS (exit 1) with no dataset row. It now SUCCEEDS (exit 0) and writes ONE free status row: `recordType: "status"`, `status: "error"`, `statusRow: true`, `uid: "status:allBoardsFailed"`, `reason: "allBoardsFailed"`, `message`, `demoMode`, `boardsRequested`, `boardsFailed`, `boardsBlocked`, `boardsNetworkError`, `boardsNotFound`, `boardsSkipped`, `failedBoards` (first 50: `input`, `status`, `error`) and `scrapedAt`. This mirrors it-hiring-signals' `allBoardsFailed` row, which also exits 0. - The message (also the terminal status message, cut to 1000 characters) reads `Not charged: every board failed or was blocked ( of entries: ). No jobs were returned and nothing was charged. Check the board URLs or slugs, or try again later. Reasons: `, plus the existing notes on free "warning" and "unresolved" records when those were written. The counts use `blocked (private, loopback or link-local address)`, `network or HTTP error`, `not found (no board)` and `skipped (robots.txt or rate-limited host)`. - Charging: the row is pushed with no charged event, so nothing is charged (as before, when the run failed). Free `unresolved` and rate-limit `warning` records are still written as before. RUN_SUMMARY is unchanged. When at least one board is read, nothing changes. - README: one new Field notes bullet ("Every board failed or was blocked"). Files: `CHANGELOG.md`, `README.md`, `my_actor/main.py` (`all_failed_status_row`, `main`), `tests/test_gap14_ratelimit.py` (the all-boards-rate-limited test now expects the status row and a succeeded run), `tests/test_gap17a_all_failed_status_row.py` (new). ### Unreleased: private-address guard (GAP17) - On the Apify platform (`APIFY_IS_AT_HOME` set), every request now checks the host first: if any address it resolves to is private, loopback, link-local, reserved, multicast, unspecified or otherwise not global (also IPv4-mapped IPv6), the request is not sent. This covers careers-page URLs, every redirect hop (an httpx request hook on the client), robots.txt and the ATS API calls. Local runs are unchanged. - A blocked careers page gets a free `unresolved` dataset row (and a RUN_SUMMARY entry) with `not fetched: private, loopback and link-local addresses are not allowed`. Nothing is charged for it. A site whose robots.txt redirects to a private address is skipped as "robots.txt unreadable". A host whose DNS lookup fails reads `DNS lookup failed for ` (same handling as before). - No change to prices, events, input, output fields or `.actor/` files. Files: `CHANGELOG.md`, `README.md` (one FAQ entry), `my_actor/sources.py`, `my_actor/main.py`, `tests/test_ssrf_guard_1102t.py`. ### Unreleased: input keys the Actor never reads no longer turn the demo into a real run (GAP16b) - **Bug fix (demo detection):** an input key that is not in the input schema and is not one of the legacy keys the code still reads (`maxTotalJobs`, `postedWithinDays`, `keywords`, `excludeKeywords`, `locations`, `remoteOnly`) is now ignored when deciding between the demo and a real run. Before, any such key (for example the Apify template's `{"helloWorld": 123}`) made a real 100-job run ($0.20); now `{"helloWorld": 1}` is the same 4-job demo as `{}` ($0.008), and an unknown key next to the prefill (`maxItems` 4) runs the same demo as the prefill alone. These keys were never read, so they change nothing else. - Unchanged: any value set for a schema key or a legacy key is still a real run under the GAP11b/GAP14 rule (for example `{"helloWorld": 1, "maxTotalJobs": 5}` gives 5 jobs, and `{"maxTotalJobs": 5}` is still a real run). - New log line (info): `Ignoring input keys that are not in the input schema: `, written once per run, only when such keys are present. - Files: `CHANGELOG.md`, `my_actor/main.py` (`SCHEMA_KEYS`, `LEGACY_KEYS`, `KNOWN_KEYS`, `unknown_keys`, `is_demo_input`, `main`), `tests/test_gap16b_unknown_keys.py` (new; checks that `SCHEMA_KEYS` matches `.actor/input_schema.json` and that every legacy key is read by the code). ### Unreleased: explicit maxItems 100 is a real run; rate-limited hosts are skipped (GAP14) - **Bug fix (demo detection):** `{"maxItems": 100}` and `{"timeRange": "7d", "maxDirectoryCompanies": 100}` were treated as the empty-input demo and stopped at 4 jobs. The input schema no longer has a `default` for `maxItems`, `maxDirectoryCompanies` and `timeRange`, so a value you type can be told apart from an empty field; the code applies the same values when they are left empty (100, 100, 7d), so `{}` behaves as before. The demo now means: nothing set, or only the prefill / `exampleRunInput` (`maxItems` 4) and the remaining schema defaults. Any other `maxItems` (100 included) is a real run. A switch set explicitly to its schema default (for example `hasSalary: false`) still counts as not set, which does not change any result. - **Rate limits (Workable 429 / Cloudflare error 1015):** a host that answers HTTP 429 is now skipped for the rest of the run (per-host circuit breaker) instead of being retried up to four times with up to 30 s waits, board after board (about 2 minutes per Workable board). Workable hosts and any Cloudflare 1015 answer trip on the first 429; other hosts get one retry when `Retry-After` is 10 s or less. Every later board on that host is skipped without a request (board status `skipped`). One free record per skipped host is pushed: `recordType: "status"`, `status: "warning"`, `host`, `reason`, `skippedBoards`, and a `message` starting "Not charged". RUN_SUMMARY gets `hostsSkipped` and the status message names the hosts. If every board failed the run still fails as before (GAP7), after writing the warning record. - **robots.txt answered with HTTP 429 is now DISALLOW** (the host is skipped with the same free warning record), no longer "no robots.txt, allow all". 404 and 410 still mean no robots.txt, and so do 401/403 (RFC 9309): `api.ashbyhq.com` answers its robots.txt with 401 (checked 2026-10-31), so treating every non-200/404 answer as disallow would skip all 487 Ashby boards. 5xx and unreadable answers were already skipped (GAP12). - Docs: README FAQ board count corrected to 692 (147 Greenhouse, 487 Ashby, 58 Lever; it said 489 Ashby), the empty-input and rate-limit behaviour described, and a note that `workArrangement` usually repeats the normalised `workplaceType` (it is inferred from title/location when the ATS gives none, so both are kept) and that Lever often publishes `team` without `department` (both fields kept as the ATS sends them; the department filter matches either). Output fields unchanged. - pricing.json (not shipped): the stale "maxItems default 1,000 / $2.00" notes now say 100 jobs / $0.20, and the board count is 692 (the 694 figure is kept as the dated 2026-10-28 measurement). A test derives the count from directory.json and checks README, schema and pricing.json. ### Unreleased: empty-input demo capped at 4 jobs (GAP11b) - **Behaviour change:** an empty input `{}` now stops at 4 jobs (4 x $0.002 = $0.008, under $0.01) instead of 100. Before, a cloud `{}` run with a $0.05 maximum cost stopped at the cost cap (25 jobs) instead of a job cap, and a local `{}` run gave 100 jobs ($0.20). The demo reads the same boards as before (the 100 largest directory boards, jobs from the last 7 days), logs a warning, and RUN_SUMMARY has `demoMode: true`. - Demo means the input sets nothing beyond the schema defaults (`{}`, or the defaults the platform fills in; empty values count as not set). Any other input is a real run: an explicit `maxItems` (other than the default 100) or any search filter keeps the normal default of 100 jobs. A run with only the defaults and `maxItems` 100 typed in is also treated as the demo, because it cannot be told apart from `{}`. - The input prefill is now only `maxItems` 4 (was 100), so the prefilled Console form runs the same 4-job demo as `{}`. The `timeRange` prefill (24h) and the `titleSearch` prefill (`engineer*`, `"data scientist"`) were removed so the form matches the demo; the `timeRange` default (7d) is unchanged. - New `exampleRunInput` in actor.json with the demo values (`{"maxItems": 4}`). ### Unreleased: a board listed twice is read and charged once; clean location strings (GAP11) - **BREAKING (fewer charged records):** the same board listed more than once in `companies` used to be read and charged once per entry (for example `airbnb` plus `https://boards.greenhouse.io/airbnb` gave every job twice and two charges per job). Entries are now matched on a normalized board key before anything is read: a slug or URL that differs only in letter case (Greenhouse, Ashby and the host-name vendors Recruitee and Teamtailor), a trailing slash, a query string or `vendor:` prefix names one board. Lever keeps letter case (its board names are case-sensitive). The board is read once, by the first entry; later entries are skipped with a log line, nothing is read or charged for them, and they are listed in RUN_SUMMARY as `duplicateEntries` (`input`, `sameBoardAs`). - A Greenhouse, Lever or Ashby job ID read through two different entries is output and charged once (those IDs are unique across boards). On other vendors an ID is only unique per board, so jobs from different boards are never merged by ID. A job already output in this run is dropped before the push, so it is never charged; RUN_SUMMARY gets `duplicateJobsDropped` and the board `duplicateJobs`, only when that happened. - **BREAKING (output values):** `location`, every `locations` entry and `country` are trimmed and runs of whitespace (tabs, line breaks, non-breaking spaces) collapsed to one space, for every ATS; a space before a comma is dropped (`Austin , TX` becomes `Austin, TX`). `United States ` is now `United States`, and `locations` no longer lists a value twice because of whitespace. An empty location becomes null. - The location filters (`locationSearch`, `locations`) now match against the cleaned values, so `United States ` in a job matches `"united states"`. - Unchanged: `uid`, the `onlyNewJobs` state and inputs that list each board once. ### Unreleased: release fixes (input validation, docs, consolidated field list) - **BREAKING (additive output fields), full list of fields added since the last release:** - Always present: `isAgency` (GAP2), `salaryAnnualUsdMin`, `salaryAnnualUsdMax` (GAP4). - With `dedupeBy: companyTitle`: `locations` (union), `duplicateCount`, `mergedUids` on merged records (GAP3). - RUN_SUMMARY, only while the matching option is on: `agencyJobsExcluded`, `duplicatesDropped`, `salaryJobsExcluded`. - `visaSponsorship`, `yearsExperienceMin`/`Max` and `status` already existed and are unchanged. - Input validation: negative numbers, a fractional count, `minAnnualSalaryUsd` above `maxAnnualSalaryUsd`, an unknown `salaryCurrency` code and unknown option values (for example `dedupeBy`, `timeRange`) now stop the run at the start with one status message listing every problem (Actor.fail, no traceback). Nothing is fetched or charged. 0 and empty keep their existing meaning (off / no limit), and `{}` plus every previously valid input behave as before. - Behaviour change: an API or scheduled input with a negative limit (for example `maxItems: -1`) used to run and now fails at the start with a status message. Nothing to read (no companies and no matching directory boards), more than 2000 companies, or an unsupported `atsFilter` value now also ends with a clean failed-run status message instead of a traceback; the run still fails as before. - Docs: README and sample output show the new fields; the README states that USD conversion uses fixed approximate rates set in October 2026 (`RATES_AS_OF`), that `isAgency`, `visaSponsorship` and `yearsExperienceMin`/`Max` are heuristics, and that the visa and years filters are off by default (this Actor already reads every description, so they add no requests here). ### Unreleased: salary in US dollars and salary filters (`hasSalary`, `minAnnualSalaryUsd`, `maxAnnualSalaryUsd`, `includeJobsWithoutSalary`) - **BREAKING (additive output fields):** every job record now has `salaryAnnualUsdMin` / `salaryAnnualUsdMax`: the annual range converted to whole US dollars with a fixed rate table bundled with the Actor (`my_actor/fx.py`, `RATES_AS_OF`). The rates are rough reference values, not live or official rates; no rate is fetched at run time. Null when there is no salary, no pay period, or the currency is not in the table (very-high-inflation currencies are left out on purpose). Consumers that reject unknown fields need updating; no existing field changes. - New inputs, all off by default: `hasSalary` keeps only jobs that state a salary; `minAnnualSalaryUsd` / `maxAnnualSalaryUsd` keep jobs whose annual USD range reaches the minimum and starts at or below the maximum; `includeJobsWithoutSalary` also keeps, under a USD bound, jobs whose salary is missing or cannot be converted. Dropped jobs are never pushed or charged. `RUN_SUMMARY` gets `salaryJobsExcluded` and each board `salaryExcluded`, only while a salary filter is on. The existing `minAnnualSalary` / `salaryCurrency` filters (posting currency) are unchanged and can be combined. - With the filters off, which jobs are output, the state store and the cost of `{}` are unchanged. ### Unreleased: duplicate removal (`dedupeBy`, opt-in) - New input `dedupeBy` (default `none`). `job` drops a job whose company, title and first location (the `dedupeKey` recipe) were already output in this run; the first one read wins. `companyTitle` merges the jobs of one company with the same title into one record: `locations` is the union of their locations, plus the new fields `duplicateCount` (jobs merged in) and `mergedUids`. Merged records are output when their board has been read; a later board of the run with the same company and title is dropped. - Duplicates are dropped or merged before the dataset push, so they are never charged; the only charged push is one per output record. `RUN_SUMMARY` gets `duplicatesDropped` and each board `duplicates`, only while the option is on. - **BREAKING only when opted in (additive output fields):** with `companyTitle`, records carry `duplicateCount` and `mergedUids` and `locations` can list more than one job's locations. With `none` (the default) output, state and the cost of `{}` are unchanged. - With `onlyNewJobs`, dropped and merged jobs are remembered as output. With `includeExpired`, a merged record is reported expired only when none of its members is listed. - Limits: only duplicates within one run are handled (a duplicate of a job output by an earlier run is output again unless `onlyNewJobs` is on). In `companyTitle` mode `maxJobsPerCompany` counts merged records. ### Unreleased: recruiting-agency flag and filter (`isAgency`, `excludeAgencies`) - **BREAKING (additive output field):** every job record now has `isAgency` (`true` / `false`). It is `true` when the company name looks like a staffing or recruitment agency (generic words such as Staffing, Recruitment, Executive Search, Personalberatung, Zeitarbeit, or a short list of known agencies such as Randstad, Adecco, Hays, Robert Half, TEKsystems) or the description says the job is filled for a client ("on behalf of our client", "our client, a …", "für unseren Kunden … suchen wir", "im Auftrag unseres Mandanten", "pour le compte de notre client", "voor onze opdrachtgever"). Consumers that reject unknown fields need updating; nothing else in the record changes. - New input `excludeAgencies` (default `false`) leaves those jobs out. They are dropped before the dataset push, so they are never charged. `RUN_SUMMARY` gets `agencyJobsExcluded` and each board `agencyExcluded`, only while the option is on. - It is a heuristic (module `my_actor/agency.py`): an in-house job at an agency's own office is also flagged, and an agency with a neutral name and no client wording is not. - With `excludeAgencies` off, which jobs are output, the state store and the cost of `{}` are unchanged. ### Unreleased (2026-10-29) - Default `maxItems` 100 (about $0.20 and under 10 seconds for `{}`), so every `{}` run stays under $0.25. - Tooling (not part of the Actor build): `tools/refresh_directory.py` re-validates the bundled directory (dropping closed boards, reporting empty and unreachable ones) and can add boards from the ATS vendors' published customer lists, writing a diff report. `.github/workflows/refresh-directory.yml` is a monthly example that opens a PR. - Greenhouse `workplaceType` / `employmentType` (and the `*Raw` fields) are read from the company's custom job-board fields (`Workplace Type`, `Employment Type`, `Time Type` and similar names). Before, they were always null on Greenhouse. Boards that publish no such field (e.g. GitLab) still give null. `workArrangement: remote` filters now also match Greenhouse jobs whose workplace field says Remote. ### 0.1 (2026-10-29, not published) - First version of Career-Site Jobs Feed, built from the ats-jobs-scraper engine (copied; that Actor is unchanged). - Feed search: `timeRange`, title / location / description / company search and exclusion with OR / AND / prefix / phrase syntax, `atsFilter`, work arrangement, employment type, seniority, minimum annual salary, visa sponsorship and maximum years of experience. - With no companies given, the built-in directory (about 690 Greenhouse, Ashby and Lever boards) is searched; company-name and ATS filters are applied to it before any request. Directory boards that have closed appear in RUN_SUMMARY with status `gone`, not as dataset records. - Enrichment: `visaSponsorship` with evidence, `yearsExperienceMin`/`Max`, `keySkills`. - `status` field; free `expired` records for jobs that left a fully read board (`includeExpired`). - `maxItems` (old `maxTotalJobs` still accepted), plus hidden aliases `keywords`, `excludeKeywords`, `locations`, `remoteOnly`, `postedWithinDays`. - Supported ATSs: Greenhouse, Lever, Ashby, Workable, Recruitee, Teamtailor. Workday, SmartRecruiters and Personio inputs are refused without a request.