# Changelog of Workday, SmartRecruiters, Recruitee, Personio Jobs Scraper (`nekruz/ats-jobs-scraper-2`) Actor - **URL**: https://apify.com/nekruz/ats-jobs-scraper-2/changelog.md - **Full Actor documentation**: https://apify.com/nekruz/ats-jobs-scraper-2.md ## Changelog: ats-jobs-scraper-2 ### 2026-10-13 - setup (Fable) - Initial version built and smoke-tested locally against live endpoints. ### 2026-10-13 - iteration 2 (reliability + cost) - **Bug fix: Workday boards were silently truncated to 40 jobs.** Workday's cxs API returns `total` only on the first page (later pages say `total: 0`), so the stop check ended pagination after page 2. NVIDIA reports 2000 jobs; the Actor returned 40. The total is now taken from page 0, and paging also ends on a short page. - **Cost:** Workday and SmartRecruiters pagination now stops as soon as `maxJobsPerCompany` new jobs pass the filters, instead of downloading the whole board (NVIDIA capped at 3: 1 request instead of ~100 after the fix; Ubisoft2 capped at 45: 1 request instead of 4). - OUTPUT `boards[].jobsOnBoard` now reports the board's real total; new `jobsFetched` shows how many were downloaded. - Test added: `workday paginates past page 2 and paginated boards stop at the cap` (asserts 45 unique Workday jobs, exactly 3 Workday list requests and 1 SmartRecruiters list request). Confirmed failing on the old code ("got 40"). Added `test/fetch-recorder.js` and the `recordRequests` option in `test/helpers.js` (from iteration 1). ### 2026-10-13 - iteration 7 (reliability + cost) - **Bug fix: `postedAfter` returned 0 Workday jobs and paged the whole board.** Workday's list API has no dates, only `postedOn` text ("Posted Today", "Posted 3 Days Ago", "Posted 30+ Days Ago"), so every Workday job failed the date filter while pagination ran to the end (NVIDIA: up to 100 list requests, 0 results; confirmed on the old code: "100 fetched, 0 after filters"). The filter now reads that relative age when no exact date is known (day precision, one day of slack; "30+ days" jobs are kept only if postedAfter is more than ~29 days back). - **Cost:** Workday lists newest first, so with `postedAfter` pagination stops once a whole page is more than 2 days older than the cutoff. The stop only applies while every page has been no newer than earlier ones (2-day jitter allowed), so a board that is not date-sorted is still paged in full. NVIDIA, last 4 days: 17 list requests, 220 jobs (was ~100 requests, 0 jobs). - With `includeDescription`, the exact Workday date from the detail page is re-checked against `postedAfter`. - Test added: `workday postedAfter uses relative post dates and stops paging at older jobs` (NVIDIA, last 5 days: results > 0, none older than the cutoff, list requests <= 35 and below a full scan). ### 2026-10-13 - iteration 12 (cost + reliability) - **Bug fix: pay-per-event charge limit was checked only after all the work.** Boards were paged and Workday / SmartRecruiters descriptions fetched for every matching job (up to the whole board without `maxJobsPerCompany`), then `Actor.charge` was asked for all of them, and the SDK charged one job past the user's limit. Now the remaining 'job' budget is checked before each board (a spent budget makes no request and a skipped bare slug is not reported as `not_found_on_any_platform`), Workday/SmartRecruiters pagination stops once the budget (or cap) of new jobs pass the filters, detail requests stop once the budget/cap is filled, and jobs are trimmed to the budget before charging. NVIDIA + Ubisoft2, descriptions, cap 30, $3 budget: 2 list + 30 detail requests and 4 jobs charged -> 1 + 3 and 3, SmartRecruiters skipped. $0.5 budget: 0 requests (was 2 list + 30 detail + 1 job charged). - With `includeDescription` + `postedAfter`, a Workday job dropped by the exact detail date no longer leaves its slot empty: detail requests continue down the list until the cap is filled. - Test added: `charge limit caps list and detail requests and never over-charges` (PPE simulated via `ACTOR_TEST_PAY_PER_EVENT` + `ACTOR_MAX_TOTAL_CHARGE_USD`; counts distinct detail URLs). Same input on the old code gave 4 charged jobs, 2 list and 30 detail requests. `runActor` gained an `env` option. ### 2026-10-13 - iteration 17 (cost + speed) - **Cost: SmartRecruiters with `postedAfter` paged the whole board.** The postings list is sorted by `releasedDate`, newest first (verified live on Ubisoft2 and BoschGroup: 0 order inversions), so every page after the first older job was wasted. Pagination now stops once a page reaches a job released before the cutoff. The stop applies only while every date seen so far has been newest-first; any out-of-order date falls back to a full scan. BoschGroup (4953 jobs), last day: 2 list requests, 168 jobs (old code: 50 list requests for the same result). Ubisoft2: 4 -> 2. - Test added: `smartrecruiters postedAfter stops paging at the first older page and loses nothing` (full Ubisoft2 scan picks a cutoff inside page 2; the filtered run must make exactly 2 list requests and return the same jobs as filtering the full scan, +/-3 for live changes). Old code: 4 list requests, fails. npm test 8/8 in ~64 s. ### 2026-10-13 - iteration 22 (reliability + speed) - **Bug fix: HTTP 429 ignored `Retry-After`.** `request()` retried a rate-limited (429) or overloaded (5xx) response after a fixed 1 s, then 2 s, so a board or detail request inside a rate-limit window usually failed all 3 attempts, and it slept another 3 s after the final failed attempt. It now waits for the server's `Retry-After` (delay-seconds or HTTP-date, never below the 1.5 s x attempt backoff, capped at 60 s) via `src/retry-after.js` (copied verbatim from seek-jobs-scraper, as ats-jobs-scraper did in iteration 16), cancels the unread body of a retried response, logs 429 retries as warnings with the requested wait, and does not sleep after the last attempt. 404 / other 4xx unchanged. - Without `Retry-After` the backoff is now 1.5 s / 3 s (was 1 s / 2 s / 3 s including the trailing sleep): a request that fails every attempt costs 4.5 s instead of 6 s. - Tests added: Retry-After unit test and `429 waits for Retry-After (seconds and HTTP-date) and never sleeps after the last attempt` (faked 429 on a Workday list POST with `Retry-After: 3`, on a SmartRecruiters detail request with an HTTP-date, and persistent 429 on Recruitee; ~6 live requests, ~18 s). Fails on the old code ("waited 1002 ms"). Ported `test/fetch-recorder.js` / `test/helpers.js` from ats-jobs-scraper (`failRequests`, `requestTimes`). npm test 10/10 in ~89 s. ### 2026-10-13 - iteration 27 (reliability + cost) - **Bug fix: a failed Workday / SmartRecruiters detail request was charged as a full job.** The job was pushed with null descriptions, no error field, and only a per-board "N of M detail requests failed" warning; under pay-per-event it was charged in full and the status said "Done: 2 jobs from 1 boards". Verified before the change: 503 on every NVIDIA detail request, cap 2, simulated PPE -> 2 jobs charged without description. - `request()` marks errors that survived every attempt on 429/5xx/network as `retryable`. Enrichers now return `{ ok }` or `{ retryable }` and leave `descriptionError` on the job (`HTTP 404 (job removed)`, `HTTP 503`, ...). Retryable failures get one deferred retry after a 5 s pause at the end of the board, which stops at the first job that still fails; 404/4xx are not retried. A recovered Workday job is re-checked against `postedAfter` with its exact date. A job still without description is pushed with `descriptionError` and not charged; only complete jobs are trimmed to the charge budget. Detail requests no longer sleep after the last one. - OUTPUT gained `jobsCharged`, `descriptionFailures`, `descriptionsRecovered` and per-board `descriptionFailures`; warning + final status message "N without description (not charged), see descriptionError". - Test added: `failed detail request is retried once later, then delivered uncharged with descriptionError` (~34 s, detail responses faked, 3 live list requests): Workday 503 x3 -> recovered after >=5 s, 4 detail requests, 1 charged; SmartRecruiters persistent 503, cap 2 -> 9 detail requests, 2 items with `descriptionError`, 0 charged, status text checked; Workday 404 -> 1 request, 0 charged. Fails on the old code. npm test 11/11 in ~115 s. - Free incomplete jobs still count toward `maxJobsPerCompany`. README does not mention `descriptionError` (out of scope). ### 2026-10-13 - iteration 32 (cost + reliability) - **Bug fix: a board entered twice was fetched twice and, with a cap, charged twice.** Jobs were deduped, but jobs past the cap were already marked seen, so the second entry paged on to new jobs: `smartrecruiters:Ubisoft2` + `https://jobs.smartrecruiters.com/Ubisoft2`, cap 2, simulated pay-per-event -> 3 list requests, 4 jobs saved and charged (reproduced live before the change). Boards are now keyed by platform + slug, case-insensitively (verified live that SmartRecruiters company ids and Workday site names are case-insensitive; the others are hostnames). A repeat (prefix vs URL, other case, trailing slash, scheme-less URL, the same bare slug again) is not requested and appears in OUTPUT as `{ status: 'duplicate', duplicateOf: '' }` with a warning. A guessed platform already tried and not found is skipped silently. Same live input now: 1 request, 2 jobs charged. - Career-site URLs without `https://` (`x.recruitee.com/`, `jobs.smartrecruiters.com/X`, `t.wd5.myworkdayjobs.com/Site`) were guessed as bare slugs on all four platforms (or rejected as unrecognized); they are now parsed as URLs. - **Reliability: a Workday or SmartRecruiters list page after the first that failed after retries (or returned no job array) turned the whole board into `error` and discarded the jobs already fetched.** Those pages are kept: board status `partial`, with `failedPage` and `error` ("page 2 failed (HTTP 404); kept 20 jobs from earlier pages"), a warning, and the final status message adds "(N incomplete, see OUTPUT)". Partial boards count as scraped boards. Recruitee, Personio and Breezy are single-request feeds (no paging), unchanged. - Test added: `same board entered twice is fetched and charged once; a failed later page keeps earlier jobs as partial` (all responses faked, 0 live requests, ~1 s): cap 2 + repeated board -> 1 request, 2 charged (old code: 4 / 4); 7 entries over 3 boards with a malformed SmartRecruiters page 2 and a 404 Workday page 2 -> 5 requests, 123 jobs, statuses partial/duplicate/ok/duplicate x2/partial/duplicate, status text checked (old code: 103 jobs, boards `error`). Ported the per-URL fake rules of `test/fetch-recorder.js` / `runActor({ failRequests: [...] })` from substack-scraper (existing single-rule calls unchanged). npm test 12/12 in ~1:56. - Still open: a run ending with 0 jobs gives no reason (port ats-jobs-scraper iteration 26 `filteredOut` + reasons); no circuit breaker for Workday/SmartRecruiters detail requests (port ats-jobs-scraper iteration 31); free incomplete jobs count toward `maxJobsPerCompany`; README does not mention `duplicate`, `partial`, `failedPage`. ### 2026-10-15 - pre-launch changes - **Workday keyword search:** with `keywords` set, the Actor now pages Workday's own search results (each keyword by its most selective word) when that takes fewer requests than reading the whole board. Narrow keywords on a 2,000-job board need about 10-20 requests instead of 100, and a search also reaches jobs past Workday's 2,000-job listing limit. The title filter still decides what is kept. Broad keywords, words under 3 letters and failed probes fall back to the full scan. A board read this way carries `workdaySearch` (the words searched) in its `OUTPUT` entry. - **Behaviour change:** Workday's search matches whole words and word beginnings, not fragments, so on Workday a keyword that only appears inside a title word (such as "gineer") no longer matches. README now says so. - Default run memory is now 256 MB (`defaultMemoryMbytes`); the maximum stays 1 GB. - The input form's `maxJobsPerCompany` is prefilled with 25 (API default stays 0, no limit). - Test added: "workday keywords page Workday search results, not the whole board".