# Changelog of SEEK Jobs Scraper: AU & NZ, Salary & Full Description (`nekruz/seek-jobs-scraper`) Actor

- **URL**: https://apify.com/nekruz/seek-jobs-scraper/changelog.md
- **Full Actor documentation**: https://apify.com/nekruz/seek-jobs-scraper.md

## Changelog: seek-jobs-scraper

### 2026-10-13 - setup (Fable)

- Initial version built and smoke-tested locally against live endpoints.

### 2026-10-13 - iteration 3 (loop)

- Cost/speed: a recent `postedAfter` (within the last 30 days) is now also sent to SEEK as the search API's
  `daterange` parameter (age in days + 1 day margin for the AU/UTC offset), so SEEK filters server-side; the local
  date filter stays as the exact cut. Before, `sortMode: KeywordRelevance` + `postedAfter` paged through every result
  (no early stop is possible on relevance order) and `ListedDate` fetched at least one all-old extra page.
  Live check (python / Sydney / relevance / last 3 days): 6 search requests -> 2, same or more jobs (66 vs 64).
  Verified beforehand that `daterange` results are a superset of the client-side filter (1- and 3-day windows, 0 missing).
- Note: with a recent `postedAfter`, OUTPUT `totalOnSite` now reports SEEK's count for that date window, not all-time.
- Test added: "recent postedAfter is filtered server-side, even with relevance sort" (asserts every search URL carries
  `daterange` and the page count matches `totalOnSite`; failed on old code with 6 requests). Ported `test/fetch-recorder.js`
  and the `recordRequests` option of `runActor` from ats-jobs-scraper.

### 2026-10-13 - iteration 8 (loop)

- Cost: with pay-per-event pricing, each page's jobs are now trimmed to what the user's max-charge budget still allows
  (`calculateMaxEventChargeCountWithinLimit('job')`) *before* descriptions are fetched and before charging, and no search
  page is requested once the budget is spent. Before, a whole page of descriptions (up to 100 GraphQL requests, ~45 s)
  was fetched and then mostly thrown away at the charge limit, and the SDK charged one job over the limit
  (it over-charges by one when asked for more than the budget allows). Local PPE simulation, budget 3 jobs, maxJobs 15,
  descriptions on: 15 GraphQL requests / 4 jobs charged -> 3 / 3. Budget below one job: 1 search + 1 charged -> no requests,
  `chargeLimitReached: true`. Non-PPE runs are unchanged (budget is treated as unlimited, as is an unpriced `job` event).
- Test added: "charge limit caps description requests and never over-charges" (runs with ACTOR_TEST_PAY_PER_EVENT and
  ACTOR_MAX_TOTAL_CHARGE_USD; asserts 3 GraphQL + 1 search requests, 3 items, and zero requests for a sub-one-job budget).
  Failed on old code. `runActor` gained an `env` option.

### 2026-10-13 - iteration 13 (loop)

- Reliability: HTTP 429 (and 5xx) retries now honour the server's `Retry-After` header. New `src/retry-after.js`
  (pure, dependency-free, meant to be copied into the other Actors): `parseRetryAfterMs` reads delay-seconds and all three
  HTTP-date forms (IMF-fixdate, RFC 850, zone-less asctime read as GMT; past dates give 0; negative, `1e3` or garbage give
  null), `retryDelayMs` waits max(linear backoff 1.5 s x attempt, Retry-After) capped at 60 s, so `Retry-After: 0` does
  not hammer and a huge value cannot stall the run. Before, every retry used the fixed 1.5 s / 3 s backoff, so a rate
  limit asking for longer was retried too early and page 1 failing killed the run.
- Speed: `request()` no longer sleeps after its final failed attempt (saved 4.5 s per permanently failed request, e.g.
  per failed description). The response body of a retried 429/5xx is cancelled to free the socket. A 429 retry is logged
  with the wait the server asked for.
- Tests added: "Retry-After parsing: ..." (unit test of both functions, incl. the cap) and "429 waits for Retry-After
  (seconds and HTTP-date) and never sleeps after the last attempt" (faked 429s via the fetch recorder: `Retry-After: 3`
  on the first search -> retry ~3 s later and the run succeeds; an HTTP-date 8 s ahead on the first GraphQL request ->
  description still arrives; persistent 429 -> exactly 3 attempts and the run ends straight after the last one).
  Old code: 1.5 s gap and a 4.5 s trailing sleep, so the test fails on it. `runActor` gained `failRequests`
  (regex, or `{ pattern, status, headers, times }`) and `requestTimes`; ported and extended from ats-jobs-scraper.

### 2026-10-13 - iteration 18 (loop)

- Reliability/cost: a job whose description could not be fetched is no longer silently charged as a full job.
  `request()` now marks errors that survived all attempts on 429/5xx/network as `retryable`. Descriptions that failed
  that way get one deferred retry at the end of the page, after a 5 s pause (`DESCRIPTION_RETRY_COOLDOWN_MS`), so a
  short rate-limit window can pass; the retry pass stops at the first job that still fails (SEEK down or blocking) so it
  never doubles the waste. Permanent failures (4xx, `jobDetails: null` for a removed job) are not retried.
- A job that still has no description is pushed with `descriptionHtml`/`descriptionPlain` null and a new
  `descriptionError` field (e.g. `HTTP 503`), and is **not charged** (pay-per-event): the buyer asked for full jobs.
  It still counts toward `maxJobs`. OUTPUT gained `jobsCharged` (SDK's charged `job` events, null outside PPE) and
  `descriptionsRecovered`; `descriptionFailures` now counts only final failures. The final status message adds
  "N without description (not charged), see descriptionError" and a warning is logged.
  Old code, first description 503 x3 then OK: 1 job lost its description and was charged; GraphQL down: 2 of 2 charged.
  New: description recovered (6 GraphQL requests), and 0 charged when GraphQL is down (9 requests: 2 x 3 + one 3-attempt retry).
- Test added: "failed description is retried once later, then delivered uncharged with descriptionError" (PPE simulated;
  injected 503 x3, persistent 503 and 404 on GraphQL; asserts request counts, items, OUTPUT and status message).
  \~35 s; npm test 10/10 in ~67 s.

### 2026-10-13 - iteration 23 (loop)

- Cost/speed: circuit breaker for job descriptions. After 5 descriptions in a row fail every attempt with 429/5xx/network
  (`DESCRIPTION_FAILURE_STREAK_LIMIT`), the rest of the page is not requested; those jobs go straight to the page's
  deferred retry, whose first request acts as a probe. If the probe succeeds the streak resets and the skipped jobs are
  fetched normally; if it fails, no description is requested for the rest of the run (`descriptionsStopped: true` in
  OUTPUT, a warning, and the final status message says "description requests stopped as SEEK kept failing"). Jobs keep
  being saved with `descriptionError` ("not requested: ...") and are not charged under pay-per-event. 404/removed jobs
  neither count toward nor reset the streak.
  Before, with GraphQL down every page cost 3 attempts per job plus a 3-attempt retry (303 requests and ~8 min per
  100-job page, so ~3,000 requests for maxJobs 1000). Now: 18 requests for the whole run, ~30 s.
  Checked: 7 jobs, GraphQL 503 always -> 18 requests (old code 24); 503 for the first 15 requests then OK -> 22 requests,
  all 7 descriptions recovered and charged, `descriptionsStopped: false`.
- Test added: "GraphQL down: description requests stop after 5 consecutive failures, jobs still saved uncharged"
  (~35 s; asserts 18 GraphQL requests, 7 items without description of which 2 "not requested", OUTPUT and status message).
  Failed on old code (24 requests). npm test 11/11 in ~103 s.

### 2026-10-13 - iteration 28 (loop)

- **Fixed: a search page failing after page 1 was reported as a complete run.** When page 2+ still failed after retries
  (429/5xx/network, 4xx or non-JSON), the Actor kept the earlier jobs, logged a warning, and ended with a plain
  "Done: 99 jobs of 571 matching", so a truncated run looked complete (OUTPUT had no flag). OUTPUT gained `partial`
  (false on a complete run), `failedPage` and `error` (e.g. `search page 2 failed (HTTP 503); kept 99 jobs from earlier
  pages`), a warning is logged, and the final status message says "(incomplete, see OUTPUT)". Same pattern as
  ats-jobs-scraper iteration 6 and substack-scraper iteration 14. A failed first page still fails the run.
- Reliability: a 200 response without a `data` array is now treated as a failed page instead of "no more jobs"
  (SEEK always sends an array, even for a nonsense keyword). On page 1 it used to end as a silent "Done: 0 jobs" with
  exit 0; now the run fails with "SEEK search failed: unexpected response shape (no data array)". On a later page the
  run is marked partial. A `null` JSON body no longer throws a TypeError.
- Test added: "failed later search page marks the run partial; malformed first page fails loudly" (~6 s; page 2 faked
  503 -> 4 search requests, page 1 jobs kept, OUTPUT partial/failedPage/error and status message checked; page 1 faked
  200 `{"error":"maintenance"}` -> 1 request, non-zero exit, clear error). The first test now also asserts a complete
  run has `partial: false`. Failed on old code. Ported the `body` option of test/fetch-recorder.js from
  ats-jobs-scraper (`failRequests.body`, string or JSON). npm test 12/12 in ~107 s.

### 2026-10-13 - iteration 33 (loop)

- **Bad input fails loudly instead of giving a silent or wrong run** (API callers bypass the editor's schema checks).
  `maxJobs` must be a whole number >= 1 (old: 0 or -5 ended as "Done: 0 jobs" with exit 0; a string compared oddly);
  `country` must be AU or NZ, case and spaces ignored (old: "UK" silently searched seek.com.au); every `workTypes` entry
  must be a known type (old: "fulltime" was dropped, so the search silently widened to all work types). Each error names
  the field and the value received; no request is made. `null` keywords/location/maxJobs/country now mean the default
  (old: `keywords: null` was sent to SEEK as the text "null").
- **A run that saves no jobs now says why.** Status was a bare "Done: 0 jobs" (or "Done: 0 jobs of 0 matching").
  Now a warning and the terminal status give the reason, e.g. "Done: 0 jobs. SEEK reported 0 matching jobs for
  "python" in "Sydney" (casual) (see OUTPUT)", "SEEK did not recognise the location "Qwertyville"" (SEEK answers
  `location: null` for an unknown place, checked live; known places incl. "All Australia" give an object), "All 3 jobs
  SEEK returned were dropped (3 listed before postedAfter 2026-01-01)", "The maximum total charge was reached before the
  first job (no request made)", "SEEK reported N matching jobs but returned an empty page", or the partial-run error.
  OUTPUT gained `filteredOut: { postedAfter, duplicates }` (jobs dropped client-side, every run). Same pattern as
  ats-jobs-scraper iteration 26. Runs that save jobs keep the old status message.
- Checked from candidate 3: a GraphQL 200 with `errors` and no content already becomes a non-retryable
  `descriptionError` (job saved, not charged); no change needed.
- Tests added: "bad maxJobs, country or workTypes fail loudly before any request" (5 inputs, 0 requests each) and
  "a run that saves no jobs says why (faked SEEK answers, no live requests)" (no match on NZ with work type, unknown
  location, all jobs older than postedAfter -> exact status text and `filteredOut`). The live empty-search test and
  the zero-budget PPE run now assert their status reasons too. Both new tests fail on the old code (exit 0 / no reason).
  npm test 14/14 in ~111 s.

### 2026-10-15 - pre-launch changes

- **Retitled** to "SEEK Jobs Scraper: AU & NZ, Salary & Full Description": the old title collided with an existing Store
  listing (actor.json, README heading and the `publish.sh` Store copy updated).
- Default run memory is now 256 MB (`defaultMemoryMbytes`); the maximum stays 1 GB.