# Changelog of Workday Jobs Scraper: Any myworkdayjobs.com Career Site (`fecklesspleb/workday-jobs-scraper`) Actor

- **URL**: https://apify.com/fecklesspleb/workday-jobs-scraper/changelog.md
- **Full Actor documentation**: https://apify.com/fecklesspleb/workday-jobs-scraper.md

## Changelog

### 2026-11-14 r3: competitor price table removed from the README

- **README only (Mike 2026-11-14):** the "How it compares" table of competitor Actors and their listed prices is replaced by a short feature list with no prices and no competitor names. Links to our own Actors are kept. No code, schema, price, default or output change.
- New guard test `tests/test_1114_no_competitors.py` (no third-party Actor links, no known competitor names, no competitor price-table wording).

### 2026-11-14 r2: fallback demo boards; copy wording (next build of 1.2)

- **Demo fallback (W3):** `{}` and the prefilled sample input (demo mode) no longer depend on one Workday site. When the demo site (https://utaustin.wd1.myworkdayjobs.com/UTstaff) gives no job (an error, a robots.txt skip such as Workday's HTTP 503 `ERR_TENANT_OUTAGE`, or 0 jobs), the demo tries, in order: https://ubc.wd10.myworkdayjobs.com/ubcstaffjobs, https://salesforce.wd12.myworkdayjobs.com/External_Career_Site, https://td.wd3.myworkdayjobs.com/TD_Bank_Careers (other Workday data centres; each robots.txt allows its site path and each returned jobs on 2026-11-14 9:21 AM ET). Boards are read one at a time and share the demo's 4-job cap, 40 s request budget and 50 s limit; a fallback is not started with under 8 s of budget left. A board that was read but whose matching jobs were only "not new" (onlyNewJobs / datasetId) stops the chain.
- **RUN_SUMMARY (demo runs):** `demoBoard` is now the board actually used (`null` if none gave a job; before, always the primary board); new `demoPrimaryBoard`, `demoFallbackUsed`, `demoFallbackReason` (when the primary gave no job) and `demoBoardsTried` (board, status, found, pushed, reason). The terminal status message ends with `Demo board: <url>.` or `Used fallback demo board <url> because <primary>: <reason>.`
- **All sample boards down:** unchanged convention (GAP12): SUCCEEDED, one free status row (no charge), now `Demo mode: no sample board gave a job (N of 4 tried: <board>: <reason>; ...). Nothing was charged. ...` with `boardsTried`; before: `Demo mode: the sample board could not be read (...)`. Safest choice: failing would mark the Store test failed and an empty dataset is not allowed; trying unlisted boards would be unbounded.
- **Demo, jobs read but none new:** message now says the board was read and its matching jobs were not new (before: "could not be read"; was SUSPECTED W5).
- **Non-demo runs:** unchanged; never use a fallback board (tested).
- README (Demo mode paragraph and FAQ) and the `companies` field description mention the fallback.
- **Copy (Lex's rules, Mike 2026-11-14):** no 'official', 'monitor(ing)' or 'alert' wording left in README, `.actor/actor.json` or `.actor/input_schema.json`. 'Monitoring mode' is now 'only-new mode (`onlyNewJobs`)' on scheduled runs; the `onlyNewJobs` field title is 'Only new jobs' and the section caption 'Only-new runs and output'; 'not an official Workday product' reads 'not a Workday product'. Text only: no field name, default, prefill, price or output change. New guard test `tests/test_1114_copy_words.py`.
- Tests: new `tests/test_1114_demo_fallback.py` (11), `tests/test_1114_copy_words.py` (3 + 1 skipped); `tests/test_gap8_demo.py::test_broken_demo_board_fails_soft` updated to the new all-down message.
- Files: `my_actor/main.py`, `README.md`, `.actor/input_schema.json`, `CHANGELOG.md`, `tests/test_gap8_demo.py`, `tests/test_1114_demo_fallback.py` (new), `tests/test_1114_copy_words.py` (new).

### 2026-11-14: contact redaction in descriptions; clearer robots.txt-unreadable message (next build of 1.2)

- **New input `redactContacts` (default on):** personal e-mail addresses, phone numbers with 8 or more digits and 'Recruiter: <name>' / 'Contact person: ...' lines in `descriptionText` / `descriptionHtml` are replaced with `[email removed]`, `[phone removed]`, `[contact removed]`. Shared role mailboxes (jobs@, careers@, accommodations@ ...) and dates are kept. Same rules and code as ats-jobs-scraper and remote-jobs-aggregator: new file `my_actor/contacts.py`, byte-identical to remote-jobs-aggregator's (sha256 86f835f9...). Why: the same Workday job read through ats-jobs-scraper was redacted but this Actor returned it verbatim (Ranger's 2026-11-10 local run of a QVC board had a non-role e-mail address in 3 descriptions).
- **Behaviour change:** with descriptions on (the default), description text can now differ from before where it contains such contact details. `redactContacts: false` gives the old text exactly. No other field changes; keys, `uid`, salary parsing (runs before redaction) and charging are unchanged.
- **Fix (message only):** a board skipped because its robots.txt could not be read (HTTP 429/5xx or timeout; the board is still skipped, as before) now reads `robots.txt unreadable on <host>: HTTP 503` in `RUN_SUMMARY` `boards[].error`, in the demo/status item that quotes it, and in the log. Before, it read `robots.txt disallows robots.txt unreadable on <host>: HTTP 503` and the log said `robots.txt disallows access`. Seen live 2026-11-14 9:07 AM ET, when utaustin.wd1.myworkdayjobs.com answered robots.txt with HTTP 503 `ERR_TENANT_OUTAGE`. A board that robots.txt really disallows keeps the same `error` text; its log line now names what is disallowed (`<board>: robots.txt disallows <path>, skipped`) instead of `robots.txt disallows access, skipped`. Same rule as remote-jobs-aggregator (`robots_skip_reason`).
- Files: `my_actor/main.py`, `my_actor/contacts.py` (new), `.actor/input_schema.json` (one new field), `README.md` (one input-table row), `CHANGELOG.md`, `tests/test_1114_redact_contacts.py` and `tests/test_1114_robots_reason.py` (new).

### 2026-11-13: chain dataset readable under limited permissions (next build of 1.2)

- **Input schema:** `datasetId` now declares `"resourceType": "dataset"` and `"resourcePermissions": ["READ"]` (editor stays `textfield`, field name and type unchanged). Apify then grants a limited-permissions run read access to the dataset given there. Fixes the chain run that failed with `ForbiddenError ... Insufficient permissions` on `GET /v2/datasets/<id>` (2026-11-13 cloud smoke of career-site-jobs-feed 0.3.1). The field description now says so and suggests `{{resource.defaultDatasetId}}` for integrations.
- **Code (`my_actor/jobschema.py`):** reads the dataset through the declared `datasetId` field as before; the integration payload `resource.defaultDatasetId` fallback is kept. When the dataset cannot be read, the error message now says which of the two it came from, and for the payload it says to put `{{resource.defaultDatasetId}}` in `datasetId`. Same error type, same point in the run, nothing fetched or charged.
- **README:** "Chain with other jobs Actors": `datasetId` row mentions `{{resource.defaultDatasetId}}`; one new bullet on read-only access under limited permissions.
- No change to output, charging, prices, defaults, prefill or runs without `datasetId`.

### 2026-11-13: README rebased onto live 1.1.2

- README only. Restored the live Related Actors wording: ATS Jobs Scraper covers 9 systems including Teamtailor, Polymer and HireHive; Remote Jobs Aggregator covers Remote OK and Jobicy. The "Chain with other jobs Actors" section is unchanged. No code, schema, input or price change.

### 2026-11-11: chain with other jobs Actors (datasetId), shared job format (version 1.2)

- **New optional inputs `datasetId`, `datasetMode`, `datasetMatchBy`, `suiteOutput`** (input-schema section "Chain with other jobs Actors (advanced)", no schema defaults, no prefill). Without them the run is unchanged: same items, same field order, same values, same charging, same `RUN_SUMMARY` keys (checked against golden output recorded from version 1.1 in `tests/fixtures/suite_golden_current.json`).
- **`datasetId`:** job items of that dataset are read (in pages, at most 100,000 items) before any board; a job already there (by `uid`, or also by `crossSourceKey` with `datasetMatchBy` "uidOrCrossKey") is dropped before the push and never charged. `datasetMode` "merge" (default) copies the earlier job items first, free (pushed without an event name); "exclude" copies nothing. Status items and expired/unresolved records are not copied. Falls back to the Apify integration payload `resource.defaultDatasetId` when `datasetId` is empty. `maxTotalJobs` counts only new jobs.
- **Unreadable `datasetId`, unknown `datasetMode` or `datasetMatchBy`:** the run fails with an input error before any board is read (as for other invalid input).
- **`suiteOutput: true`:** each job item is rewritten into the shared field order (`my_actor/jobschema.py`, `SUITE_FIELDS`), with missing shared fields as null or \[]; values unchanged; jobs copied from `datasetId` are kept as they were. Off by default.
- `RUN_SUMMARY` gets `previousDataset` (only when `datasetId` is used) and `suiteOutput` (only when true).
- New shared file `my_actor/jobschema.py` (JSON Schema `jobs-suite/1`, byte-identical in all four jobs Actors). Tests: `tests/test_1111_suite_contract.py`, `tests/test_1111_suite_chain.py`, `tests/suite_mock.py`, fixtures `jobs-suite-item.schema.json`, `suite_golden_current.json`, `suite_shared.sha256.json`.
- README: section "Chain with other jobs Actors". No change to prices, `requirements.txt` (`apify >= 4.1, < 4.2`), the Dockerfile, `exampleRunInput` or the prefill.

### 2026-11-10: dedupeKey and crossSourceKey on every job (version 1.1)

- **New output fields** on every job record: `dedupeKey` (company + title + first location, the ATS Jobs Scraper recipe; the same value `dedupeBy: "job"` already compared, so that option behaves as before) and `crossSourceKey` (company with its legal form removed + title, no location; `null` when either is empty). Recipe in `my_actor/jobkey.py`, byte-identical in ats-jobs-scraper and remote-jobs-aggregator, so the same job gets the same keys in all three.
- `dedupe.job_key` now uses `jobkey.dedupe_key`; values unchanged. `uid`, charging, inputs and state are unchanged.
- README: API example uses `/v2/actors/` from one `APIFY_API` variable; rows for the two new fields.
- Rebased onto live build 1.0.23 (UT Austin UTstaff is already the demo board, prefill and `exampleRunInput`).
- Changed files: `.actor/actor.json` (version 1.1), `CHANGELOG.md`, `README.md`, `my_actor/dedupe.py`, `my_actor/jobkey.py` (new), `my_actor/main.py`, `tests/fixtures/jobkey_vectors.json` (new), `tests/test_1110_jobkey.py` (new), `tests/test_1110_keys.py` (new). No change to inputs, `requirements.txt` (`apify >= 4.1, < 4.2`), the Dockerfile or prices.

### 2026-11-10: sample-board URLs in error messages and tests (UT Austin UTstaff)

- User-facing error examples in `my_actor/sources.py` and the comments next to them now use https://utaustin.wd1.myworkdayjobs.com/UTstaff, the current demo board, instead of the previous NVIDIA sample URL. Historical CHANGELOG entries are unchanged.
- Tests that used that NVIDIA URL, including live tests that hit NVIDIA when `WORKDAY_LIVE_TESTS` is on, now use the UT Austin UTstaff board (and a robots.txt fixture fetched 10 Nov 2026). Company-code stripping examples in tests use generic names (`2100 Example Corp`).
- Changed files: `CHANGELOG.md`, `my_actor/sources.py`, `tests/test_1103_tester.py`, `tests/test_gap11_duplicates_locations.py`, `tests/test_robots_live_path.py`, `tests/test_workday_facet_split.py`, `tests/test_workday_hosts.py`, `tests/test_workday_nodesc.py`, `tests/test_workday_robots.py`, `tests/run_ppe.sh`, `tests/fixtures/utaustin.wd1.robots.txt` (new). The NVIDIA robots fixture is no longer in the tree.

### Unreleased: sample board moved to UT Austin (Nov 10, 2026)

- The demo and sample board (`DEMO_BOARD` in `my_actor/main.py`), the `companies` prefill and its schema description, and `exampleRunInput` in `.actor/actor.json` now use https://utaustin.wd1.myworkdayjobs.com/UTstaff instead of the previous sample board. The demo logic, caps (4 jobs, $0.008), time limits and every other default are unchanged; the prefilled input still runs as the demo, and `{}` still runs the demo, now on the new board.
- README: the sample input, example inputs, API snippets and sample output items now use the new board (examples re-run locally on November 10, 2026: 4, 6 and 5 jobs). The schedule section says **Posted within last N days**, the field's label.

### Unreleased: state store errors and no empty dataset (GAP16)

- A state store that cannot be opened (an invalid `stateStoreName` such as `Bad Name!!`, a store owned by another account, limited permissions) used to crash the run with a traceback (exit 91) and an empty dataset when `onlyNewJobs` or `includeExpired` was on. Now it logs a warning and the run goes on as a first run with nothing saved; RUN_SUMMARY gets `stateStoreUnavailable: true` and the status message says so (same as the rss, sitemap and remote Actors).
- When no board could be read and nothing was written (for example robots.txt could not be fetched, so the board was skipped), the run used to end SUCCEEDED with an empty dataset. It now pushes one free `status` item (no charged event). The exit status is unchanged.
- GAP16 changed these files: `CHANGELOG.md`, `my_actor/main.py`, `tests/test_gap16_store_and_empty_dataset.py`, `tests/test_gap8_demo.py` (`test_broken_user_board_still_fails_loudly` now expects the free status row). No change to `.actor/input_schema.json` or `.actor/actor.json`.

### Unreleased: prefilled sample input, README examples (GAP15)

- **Input form prefill:** `companies` is prefilled with the demo site (https://nvidia.wd5.myworkdayjobs.com/NVIDIAExternalCareerSite) and `maxTotalJobs` with 4, so a Store visitor sees a real example instead of an empty form. No schema `default` was added: `companies` still has no default and `maxTotalJobs` still defaults to 0, so an API run with `{}` is unchanged (the GAP8 demo).
- **The prefilled input runs as the demo:** new `is_sample_input()` in `my_actor/main.py`. When `companies` is exactly the demo site (a trailing `/` is ignored) and `maxTotalJobs` or `maxJobsPerCompany` is between 1 and 4, the run takes the demo path (`demo_input()`, the demo cap and time limit, `RUN_SUMMARY.demoMode`). The log says `Demo mode: prefilled sample input, returning up to 4 sample jobs from https://nvidia.wd5.myworkdayjobs.com/NVIDIAExternalCareerSite` and the status message starts `Demo mode (prefilled sample input): `. So the Store test input costs what `{}` costs: at most 4 × $0.002 = $0.008. Any other site, a second entry, or a cap of 0 or above 4 is a normal run, exactly as before.
- `exampleRunInput` in `.actor/actor.json` was `{"maxTotalJobs": 4}`; it is now the same as the prefill (the GAP12 test requires that).
- Schema descriptions of `companies` and `maxTotalJobs` mention the prefill and its cost.
- README: new "What you get" section near the top, "Input examples" (three inputs, each run locally on November 1, 2026) with a real output item from the prefilled input, a sample-run pricing line, and an FAQ entry on which dataset rows are free. The demo paragraph and input table mention the prefill.
- Tests: `tests/test_gap8_demo.py` now expects the `companies` prefill to be the demo site (it used to require no prefill; the no-default assertion stays). New `tests/test_gap15_sample_input.py` covers the prefill, the sample input taking the demo path, `{}` being unchanged, and other inputs being normal runs.
- Tests: removed the unused `datetime` imports in `tests/test_gap14_spotlight_ids.py` (pyflakes warnings). No behaviour change.
- Review fixes (GAP15b, README and this changelog only, plus the heading check in `tests/test_gap15_sample_input.py`): "What you get" is renamed "At a glance" and its first bullet names the platform; the FAQ says the status row is written when the demo or a time-limited run "ends without any job row" (the code condition is `(demo or cut) and not runner.pushed`, so it also covers a demo whose filters removed every job) and says to keep rows without `status`, or with `status: "active"`, to get only jobs.
- GAP15 changed these files: `.actor/actor.json`, `.actor/input_schema.json`, `CHANGELOG.md`, `README.md`, `my_actor/main.py`, `tests/test_gap14_spotlight_ids.py`, `tests/test_gap15_sample_input.py`, `tests/test_gap8_demo.py`. The `.actor/actor.json` version is unchanged.

### Unreleased: Workday 'Spotlight Job' IDs, README demo count (GAP14)

- **Bug fix (onlyNewJobs pre-check key):** the search-result ID was always `bulletFields[0]`. Intel lists a label first on promoted jobs (`["Spotlight Job", "JR0286800"]`), so those jobs got the ID `Spotlight Job` until the detail request replaced it with `jobReqId`. The "only new jobs" pre-check (before the detail request) therefore looked up `workday:intel:Spotlight Job`, never matched, and every known Spotlight job had its detail page requested again on every scheduled run (it was then dropped by the final check, so it was not charged twice). If the detail request failed, all Spotlight jobs on a site shared that one ID. The list ID is now the `bulletFields` entry that the job's `externalPath` ends with (`..._JR0286800`, or `..._JR2028745-1` after a repost), else the first entry that looks like an ID, else the path's tail; `requisitionId` from the list uses the same rule. Unchanged for every site whose `bulletFields` starts with the ID (NVIDIA and the other tested sites), and the final `id`/`uid` still comes from the detail record's `jobReqId`. Fixture test: `tests/fixtures/intel_spotlight_search.json`.
- Docs: the README input table said "the 5-job demo"; the demo returns up to 4 jobs (`DEMO_MAX_JOBS = 4`), as the README demo paragraph already said. The demo paragraph said the demo stops after 60 seconds; the cut-off has been 50 s since GAP12 (`DEMO_TIMEOUT_SECS`).

### Unreleased: changelog file lists (GAP13)

- GAP13 changed these files: `CHANGELOG.md`. No change to `.actor/input_schema.json` or `.actor/actor.json`.
- GAP12 changed these files: `.actor/actor.json`, `CHANGELOG.md`, `README.md`, `my_actor/main.py`, `my_actor/runguard.py`, `my_actor/sources.py`, `tests/test_gap12_example_input.py`, `tests/test_gap12_runguard.py`, `tests/test_gap8_demo.py`. In `.actor/actor.json` GAP12 only added `exampleRunInput` (shown in the GAP12 entry below); it did not change the description, the version or any other field. GAP12 made no change to `.actor/input_schema.json` (defaults, prefill and field texts are as before).

### Unreleased: fail soft and a time budget for the Store test (GAP12)

- Fail soft in the demo: when the sample site cannot be read, the demo no longer fails or returns an empty dataset. It pushes one free `status` item (no charged event) with the reason, logs a warning, and ends SUCCEEDED. A run cut short by the time budget with no job does the same. A run on your own sites where every site fails still fails, as before.
- The demo is now cut off after 50 s (was 60 s).
- Time budget: the demo run (`{}`, the prefill or the exampleRunInput) starts no new request after about 40 s, uses a 10 s timeout per request with at most 2 attempts and waits at most 2 s between them (a long `Retry-After` is capped). Every run, demo or not, also stops starting requests 20 s before the platform timeout (`ACTOR_TIMEOUT_AT`) and gets cut off at most 10 s after that, so a slow or hung source can no longer make the run time out. When the platform aborts the run, no new request is started.
- robots.txt is now fetched with one lock per host instead of one lock for all hosts, so one slow host no longer holds up the others.
- New `exampleRunInput` in actor.json: `{"maxTotalJobs": 4}`. It matches the prefill and runs the same small demo.
- Tests added for a source that is down, slow or hung, a partial failure, the time budget, the free status item and an unreadable state store.

### Unreleased: a board listed twice is read and charged once; clean location strings (GAP11)

- **BREAKING (fewer charged records):** the same board listed more than once in `companies` used to be read and charged once per entry (for example `airbnb` plus `https://boards.greenhouse.io/airbnb` gave every job twice and two charges per job). Entries are now matched on a normalized board key before anything is read: `https://nvidia.wd5.myworkdayjobs.com/NVIDIAExternalCareerSite`, the `/en-US/` form, an upper-case host, a trailing slash or a query string are one site. The board is read once, by the first entry; later entries are skipped with a log line, nothing is read or charged for them, and they are listed in RUN_SUMMARY as `duplicateEntries` (`input`, `sameBoardAs`).
- Workday job IDs are only unique per site, so jobs from different sites are never merged by ID. A job already output in this run is dropped before the push, so it is never charged; RUN_SUMMARY gets `duplicateJobsDropped` and the board `duplicateJobs`, only when that happened.
- **BREAKING (output values):** `location`, every `locations` entry and `country` are trimmed and runs of whitespace (tabs, line breaks, non-breaking spaces) collapsed to one space, for every ATS; a space before a comma is dropped (`Austin , TX` becomes `Austin, TX`). `United States ` is now `United States`, and `locations` no longer lists a value twice because of whitespace. An empty location becomes null.
- The `locations` filter ignores extra whitespace in both the input term and the job's location, so `United  States ` matches `United States`.
- Unchanged: `uid`, the `onlyNewJobs` state and inputs that list each board once.

### Unreleased: Workday pathless rows and partial reads (GAP8, Ranger's review)

- **BREAKING (output):** Workday list rows with no `externalPath` (no job page) used to be output with the bare site URL as `url`, deduped by title (so different jobs with the same title merged into one) and charged as jobs. They are now skipped: never pushed and never charged. The board's `RUN_SUMMARY` entry gets `pathlessSkipped` (distinct rows skipped), only when there were any, and the log says so.
- **BREAKING (RUN_SUMMARY):** a Workday site whose jobs could not all be reached past the 2,000-row cap now has `status: "partial"` with a `partialReason`, instead of `ok`. This now also covers sites where no facet covers every job (for example only a distance facet is left): the best facet is still used, but the read is reported as partial and the log warns, where before it could log that facet splitting reached every job. As in GAP1, such a read never reports jobs as expired. `boardsOk` and the final status message count partial boards as read. A stop by `maxJobsPerCompany` or `maxTotalJobs` is not partial.

### Unreleased: demo mode for an empty input (GAP8)

- Behaviour change: an input with no companies (and no directory search), including `{}`, used to fail at the start. It now succeeds as a demo run: up to 4 sample jobs from https://nvidia.wd5.myworkdayjobs.com/NVIDIAExternalCareerSite, charged at the normal per-job price (at most 4 job events), stopped after 60 seconds at the latest. The log line and status message start with `Demo mode`. RUN_SUMMARY gets `demoMode` and `demoBoard` only in demo runs.
- Input schema: `companies` has no prefill and is not required, so Apify's Store test (which runs the prefilled default input) and an empty API call both run the demo. Inputs that list companies behave as before.

### Unreleased: release fixes (input validation, docs, consolidated field list)

- **BREAKING (additive output fields), full list of fields added since the last release:**
  - Always present: `salaryAnnualMin`, `salaryAnnualMax`, `salaryAnnualUsdMin`, `salaryAnnualUsdMax` (GAP4).
  - With `includeExpired`: `status` on every job record (`active`), plus free `expired` records with `uid`, `id`, `title`, `url`, `companyName`, `lastSeenAt`, `expiredDetectedAt` (GAP1).
  - With `dedupeBy: companyTitle`: `locations` (union), `duplicateCount`, `mergedUids` on merged records (GAP3).
  - With their filters on: `workArrangement`, `seniority`, `visaSponsorship`, `visaSponsorshipEvidence`, `yearsExperienceMin`, `yearsExperienceMax` (GAP5).
  - RUN_SUMMARY, only while the matching option is on: `expiredRecords`, `duplicatesDropped`, `salaryJobsExcluded`, `jobFiltersExcluded`.
- Input validation: negative numbers, a fractional count, `minAnnualSalaryUsd` above `maxAnnualSalaryUsd` and unknown option values (for example `dedupeBy`, `timeRange`) now stop the run at the start with one status message listing every problem (Actor.fail, no traceback). Nothing is fetched or charged. 0 and empty keep their existing meaning (off / no limit), and `{}` plus every previously valid input behave as before.
- Behaviour change: an API or scheduled input with a negative limit (for example `maxJobsPerCompany: -1`) used to run and now fails at the start with a status message. A missing or too long `companies` list (none given and no directory search, or more than 2000) now also ends with a clean failed-run status message instead of a traceback; the run still fails as before. `salaryCurrency` is not an input of this Actor and is ignored, as before.
- Docs: README and sample output show the new fields; the README states that USD conversion uses fixed approximate rates set in October 2026 (`RATES_AS_OF`), that `visaSponsorship`, `yearsExperienceMin`/`Max`, `workArrangement` and `seniority` are heuristics, and that the visa and years filters are off by default and fetch job details, so they slow runs down.

### Unreleased: job filters from career-site-jobs-feed (`seniority`, `employmentTypes`, `workArrangement`, `visaSponsorship`, `maxYearsExperience`, `timeRange`)

- New inputs, all off by default, ported unchanged from career-site-jobs-feed (same patterns, same values): `seniority`, `employmentTypes`, `workArrangement`, `visaSponsorship` (any / yes / not_no; default `any` = off), `maxYearsExperience` and `timeRange` (1h / 24h / 7d / 30d / all). `timeRange`, when set, replaces `postedWithinDays`; undated jobs still follow `includeUndatedJobs`. Dropped jobs are never pushed or charged. `RUN_SUMMARY` gets `jobFiltersExcluded` and each site `filterExcluded`, only while one of these filters is on.
- **BREAKING only when opted in (additive output fields):** `workArrangement` (with that filter), `seniority` (with that filter), `visaSponsorship` + `visaSponsorshipEvidence` (with `visaSponsorship` other than `any`; the evidence snippet has e-mail addresses and phone numbers removed) and `yearsExperienceMin` + `yearsExperienceMax` (with `maxYearsExperience`) are added to every record while that filter is on. No existing field changes.
- With `visaSponsorship` or `maxYearsExperience` on, job details are fetched even when `includeDescription` is off (more requests, slower runs); the description is still only output when `includeDescription` is on. A salary stated only in the description can then also fill `salaryMin`/`salaryMax`.
- While any of these filters is on, `maxJobsPerCompany` is not passed to the Workday adapter, so the cap counts jobs after the filters.
- With every new input left empty, which jobs are output, the requests made, the output fields and the cost of `{}` are unchanged.

### Unreleased: salary in US dollars and salary filters (`hasSalary`, `minAnnualSalaryUsd`, `maxAnnualSalaryUsd`, `includeJobsWithoutSalary`)

- **BREAKING (additive output fields):** every job record now has `salaryAnnualMin` / `salaryAnnualMax` (the pay range as a yearly amount in the posting's own currency: hour × 2,080, day × 260, week × 52, month × 12) and `salaryAnnualUsdMin` / `salaryAnnualUsdMax`: the annual range converted to whole US dollars with a fixed rate table bundled with the Actor (`my_actor/fx.py`, `RATES_AS_OF`). The rates are rough reference values, not live or official rates; no rate is fetched at run time. Null when there is no salary, no pay period, or the currency is not in the table (very-high-inflation currencies are left out on purpose). Consumers that reject unknown fields need updating; no existing field changes.
- New inputs, all off by default: `hasSalary` keeps only jobs that state a salary; `minAnnualSalaryUsd` / `maxAnnualSalaryUsd` keep jobs whose annual USD range reaches the minimum and starts at or below the maximum; `includeJobsWithoutSalary` also keeps, under a USD bound, jobs whose salary is missing or cannot be converted. Dropped jobs are never pushed or charged. `RUN_SUMMARY` gets `salaryJobsExcluded` and each site `salaryExcluded`, only while a salary filter is on. On Workday, `maxJobsPerCompany` is no longer passed to the adapter while a salary filter is on, so the cap counts jobs after the filter.
- With the filters off, which jobs are output, the state store and the cost of `{}` are unchanged.

### Unreleased: duplicate removal (`dedupeBy`, opt-in)

- New input `dedupeBy` (default `none`). `job` drops a job whose company, title and first location (the `dedupeKey` recipe) were already output in this run; the first one read wins. `companyTitle` merges the jobs of one company with the same title into one record: `locations` is the union of their locations, plus the new fields `duplicateCount` (jobs merged in) and `mergedUids`. Merged records are output when their site has been read; a later site of the run with the same company and title is dropped.
- Duplicates are dropped or merged before the dataset push, so they are never charged; the only charged push is one per output record. `RUN_SUMMARY` gets `duplicatesDropped` and each site `duplicates`, only while the option is on.
- **BREAKING only when opted in (additive output fields):** with `companyTitle`, records carry `duplicateCount` and `mergedUids` and `locations` can list more than one job's locations. With `none` (the default) output, state and the cost of `{}` are unchanged.
- With `onlyNewJobs`, dropped and merged jobs are remembered as output. With `includeExpired`, a merged record is reported expired only when none of its members is listed. On Workday, `maxJobsPerCompany` is no longer passed to the adapter while dedupe is on, so the cap counts records after duplicates are removed.
- Limits: only duplicates within one run are handled (a duplicate of a job output by an earlier run is output again unless `onlyNewJobs` is on). In `companyTitle` mode `maxJobsPerCompany` counts merged records.

### Unreleased: expired-job tracking (`includeExpired`, opt-in)

- New input `includeExpired` (default `false`), ported from Career-Site Jobs Feed. On scheduled runs a free record with `status: "expired"` (`uid`, `ats`, `companySlug`, `companyName`, `id`, `title`, `url`, `lastSeenAt`, `expiredDetectedAt`, `scrapedAt`) is added for each job an earlier run returned that the site no longer lists. It is pushed without the `job` event, so it is never charged. New dataset view **Expired jobs**.
- Only sites read completely are checked: a read cut short by `maxJobsPerCompany`, `maxTotalJobs`, the cost limit or an abort, or a Workday site with rows past the 2,000-row cap that facet splits could not reach, reports nothing as expired. Jobs that Workday title filters or `onlyNewJobs` drop before the detail request still count as listed.
- State: a new `<key>-live` entry per site in the state store. When a job expires its seen entry is dropped, so a job that comes back is returned as new.
- With `includeExpired` on, job records get `status: "active"`, and `RUN_SUMMARY` gets `expiredRecords` plus `expired` / `complete` per site. With it off (the default), output, state and cost are unchanged, so nothing here is breaking.

### Unreleased: Workday sites on myworkdaysite.com and embedded career sites

- Workday URLs of the form `https://wdN.myworkdaysite.com/[locale/]recruiting/<tenant>/<site>` are now accepted. They are read through `https://<tenant>.wdN.myworkdayjobs.com/<site>`, the same career site and API, so the tenant's robots.txt there applies (myworkdaysite.com's own robots.txt has no rules) and job URLs in the output use the myworkdayjobs.com form.
- A company careers page that embeds a Workday site (the page's `window.workday` config names the tenant, site and data center) is now recognized, as well as pages that link to a myworkdaysite.com URL.
- Workday has no custom (vanity) domains for career sites, so nothing else changes.

### Unreleased: one vocabulary for workplaceType and employmentType

- **Breaking for exact-value filters:** `workplaceType` is now always `remote`, `hybrid`, `onsite` or null, and `employmentType` is always one of the schema.org values `FULL_TIME`, `PART_TIME`, `CONTRACTOR`, `TEMPORARY`, `INTERN`, `VOLUNTEER`, `PER_DIEM`, `OTHER`, or null, whatever the ATS. Before, each ATS's own spelling was passed through (`on-site`, `OnSite`, `Full-time`, `FullTime`, `Full time`).
- The ATS's own text is kept in the new fields `workplaceTypeRaw` and `employmentTypeRaw`.
- A workplace value that says nothing clear (Workday's `Flexible`, Lever's `unspecified`) becomes null; an employment value that fits none of the types (Lever's `Permanent`) becomes `OTHER`.

### Unreleased: robots.txt rule for every host

- If a host's robots.txt can't be read (a server error, a page that isn't a robots file, or a network failure), the whole host is now skipped for the run with a warning in the log, on every ATS. Before, only Workday hosts were skipped and other hosts were read without rules.
- A server error or network failure on robots.txt is retried once, 1.5 seconds later, before the host is skipped.
- A missing robots.txt (any 4xx) still means no rules. A readable robots.txt is obeyed as before.
- The rule also covers detection probes: company-name and slug auto-detection, board-name lookups and careers-page sniffing. A careers page that redirects to another host is now followed one hop at a time, and each host's robots.txt is checked before its page is fetched. A host skipped this way counts as no board, and the run carries on with the other hosts.

### Unreleased: monitoring keeps jobs that are still listed

- **Jobs that stay listed for more than 120 days no longer come back as "new".** With `onlyNewJobs` (or `trackFirstSeen`) on, every run now refreshes the last-seen time of each known job it finds, not only of jobs it outputs. A job is forgotten only after 120 days in which no run saw it.
- **Known Workday jobs cost no detail request.** With `onlyNewJobs` on, jobs already in the state are dropped before their detail page is read, and they no longer count toward `maxJobsPerCompany`.

### Unreleased: sites with more than 2,000 jobs

- **Large Workday sites are read in full.** Workday's job list never returns more than 2,000 rows for one query, so sites such as NVIDIA (about 2,700 jobs) were cut off at 2,000 without a warning. When a site reports 2,000 or more, the Actor now splits the query by the site's own category filters (for example job family or location) and merges the results, without duplicates.
- If some jobs still can't be reached that way, the log says how many and which filter values are still capped.

### Unreleased (2026-10-28): fields no longer depend on `includeDescription`

#### Fixed

- **Job detail pages are always read.** Before, turning `includeDescription` off skipped them, so `employmentType`, `country`, `requisitionId`, `hiringEntity` and the extra locations came back empty. Now they are filled in either way, and `includeDescription` only decides whether the description is in the output. Turning it off no longer saves requests.
- **`companyName` is the company, not the tenant slug.** It is the hiring entity without its leading company code (`2100 NVIDIA USA` becomes `NVIDIA USA`). Before, it was the tenant name from the URL (`nvidia`). It is null only when the detail page could not be read.
- **`postedAt` is a calendar date** (`2026-10-28`, no time). Labels such as "Posted Today" are counted on the US Eastern calendar. Before, they were counted in UTC, so evening postings could show the next day. For "Posted 30+ Days Ago" the posting start date from the detail page is used.
- **Multi-location jobs** no longer show "3 Locations" as their location. The full list comes from the detail page.
- **Remote detection** also looks at the primary location in the job URL, for example `US-Remote`.
- **Pay from the description and `searchInDescription` work with `includeDescription` off,** because the description is always read.
- `postedWithinDays` compares date-only values by calendar day.

#### Changed

- Default run memory is now 1024 MB (was 512 MB).
- Added `tzdata` to requirements.txt so the US Eastern calendar works in the container.
