# Changelog of WeWorkRemotely Scraper | Remote Jobs + Full Descriptions (`tqm/weworkremotely-scraper`) Actor - **URL**: https://apify.com/tqm/weworkremotely-scraper/changelog.md - **Full Actor documentation**: https://apify.com/tqm/weworkremotely-scraper.md ## Changelog ### 2026-09-10 — the input sample advertised a category that returns nothing `docs/README.store.md`'s input sample used `"categories": ["Programming", "DevOps and Sysadmin"]`. Measured against the live board: `["DevOps and Sysadmin"]` returns **0 rows and FAILS the run**, while `["DevOps"]` returns **18**. The field table on the same page already said `"DevOps"`. Sample corrected. No behaviour change — every other filter on this Actor checked out: `minSalary` 28/28 correct, `categories: ["Programming"]` 25/25, `techStackFilter: ["python"]` 15/15. > ⚠️ **An example value in a listing is a promise that it works.** This one was two words longer than > the value that does. ### Unreleased #### 2026-09-08 — a short result set now says whose limit it was Measured against the live Store listing: asking for the advertised maximum of **300** returned **91** (30%), and the count reproduced exactly across separate runs — so this is the board's real inventory, not a flaky scrape. The status message said only `Returned 91 WeWorkRemotely postings.`, which a buyer who asked for 300 cannot tell apart from a broken Actor. `finishRun()` now receives `requested: maxJobs` and appends the reason: ``` Returned 91 WeWorkRemotely postings. That is every posting on the board matching your filters — asking for 300 will not return more today. ``` `RUN_STATS` also carries `requested` and `boardExhausted`. Logic and tests live in `apify-actor-kit` (`inventory.ts`); `src/kit/` here is generated. ### \[2.1.0] - 2026-08-27 #### Added — a zero-row run now says why, and fails Adopts the shared `src/kit/`, generated from [`apify-actor-kit`](https://github.com/Atredies/apify-actor-kit) — do not edit in place; `check-sync.sh` fails on any hand-edit. Until now a run that returned nothing printed `✓ Scraping complete!` and exited `SUCCEEDED`. A buyer paying per result could not tell a too-narrow filter from a broken actor from a change in WeWorkRemotely's RSS. Each is now a distinct, named, failing outcome carrying a per-filter breakdown, with `failOnZeroResults` (default `true`) to opt back into quiet green runs. #### Fixed — one rate-limited feed no longer discards every feed already fetched `fetchFeed()` threw on a non-OK response and the feed loop had no `catch`, so a single HTTP 403 lost **every** row from every feed already read. WeWorkRemotely does rate-limit rapid sequential requests: measured 2026-08-27, requesting all nine category feeds back to back returned **403 on two of them**. Feeds are now fetched with a 1 s politeness gap, failures are counted rather than fatal, and `blocked` reaches the diagnosis — so "every feed was refused" is a different, actionable answer from "no postings found". **Measured effect on that exact run: 0 rows → 245 rows**, with `feedsBlocked: 2` recorded in `RUN_STATS`. #### Fixed — `maxJobs` advertised a ceiling it cannot reach The schema offered up to **1,000**. WWR's feeds are single un-paginated pages: measured 2026-08-27, the default all-jobs feed carried **89** postings and all nine category feeds together **~250**. Maximum corrected to **300**, description states the real limit. Selling a cap that cannot be delivered is a refund request. #### Changed - One batched `Actor.pushData()` per run instead of one call per row. > Note: the 2.0.0 rewrite (browser → RSS, 2026-08-14) predates this changelog's > coverage and is recorded in the TQM vault's *Scraper Commercialisation Plan* §2. ### Unreleased — 2026-08-13 Documentation only. **No behavior changes**; build `0.0.4` is unmodified. #### Verified — the HTTP hypothesis is CONFIRMED `IMPROVEMENT-PLAN.md` item 4 asked whether detail pages render without JavaScript. **They do.** Plain `curl` — no browser, no proxy — returns every field `main.ts` extracts, on the listing page and on **5 of 5** detail pages: title, company, the full 648-word description, company profile URL, region, job type, and 51 job links from `section.jobs`. `CheerioCrawler` can therefore replace `PuppeteerCrawler` with no data loss, moving this actor from \~$4.19 to HTTP-class cost (~$0.05–$0.26 per 1,000) — **−323% margin to ~95%**. This was the gate on the actor's entire commercial case; it is now open. The migration itself is not yet done. #### Added - Full documentation suite: `docs/ARCHITECTURE.md`, `docs/DEVELOPMENT.md`, `docs/WWR-DOM.md`, `docs/IMPROVEMENT-PLAN.md`, and `CLAUDE.md`. - `.actor/dataset_schema.json` — the output surface was previously undeclared. - README rewritten as a buyer-facing listing with a known-limitations table. #### Verified against live WeWorkRemotely Platform run on 2026-08-13: 8 rows, $0.0335, 241 s — the run hit an imposed timeout rather than finishing naturally. Roughly **30 s and $0.004 per job**. **This is the highest-quality actor of the job-board set.** Unlike its siblings it extracts real company names, real full job descriptions and real company profile URLs. Two reasons: - It uses WeWorkRemotely's **semantic BEM class names** rather than Tailwind utility classes (as in the YC actor) or guessed `[class*="…"]` substrings (as in the Wellfound actor). - It **validates before emitting** — three gates on title, company and description length — so pages that fail to parse are skipped rather than emitted as hollow rows. No other actor in the portfolio does this. #### Findings recorded (not fixed) - **`jobId` is the company slug and collides.** It is derived by splitting the URL's final segment on the first hyphen, which yields the company, not the job. Measured: **8 rows, 6 unique `jobId`s** — three separate Toptal postings all reported `jobId: "toptal"`. Consumers must key on `jobUrl`. - **`maxJobs` does not bound the run.** The listing loop checks `processedCount >= maxJobs` before queueing, but `processedCount` only increments in the detail handler, which hasn't run — so it is `0` throughout and every discovered URL is queued. Identical to the sibling Wellfound bug; masked here because the run hit a timeout first. At ~30 s per job this is a real cost exposure. - A zero-row run exits **SUCCEEDED**. Worse, because failed rows are silently skipped, a broken selector shows up as a *low row count* rather than an error. - `category` and `postedAt` read from the same sidebar list without qualifying which item — observed `category` values mix real categories (`"All Other Remote"`) with employment types (`"Full-Time"`). `jobType` is the reliable field. - `postedAt` is relative prose (`"9 hours ago"`), not a timestamp. - `currency` is hard-coded `"USD"`; the salary regex runs over the description, so it can match an unrelated dollar figure; `max` is read by computed array index. - `description` is hard-truncated at 2,000 characters — most rows hit the cap exactly. - Validation skips are logged but never counted or surfaced. - `skills` duplicates `techStack`. All are itemized and prioritized in `docs/IMPROVEMENT-PLAN.md`, which flags one test to run early: the detail pages appear server-rendered, so `CheerioCrawler` may be able to replace Puppeteer entirely — potentially a ~30× cost reduction with no data loss. *** ### 0.0.4 and earlier No changelog was kept. Build `0.0.4` is the deployed build as of 2026-08-13; the last source commit was 2025-11-16. ### 2026-09-04 — `companyUrl` was advertised with an example and never populated Found by a pre-launch audit comparing every listing claim against the code across the portfolio. The dataset schema described `companyUrl` as *"WeWorkRemotely company profile URL"* and gave the example `https://weworkremotely.com/company/collibra`. It is declared on the row interface (`src/main.ts:50`) and **never assigned anywhere** — the field is always absent. This actor reads WWR's RSS feeds, which carry the full job description but no company profile link. The description now says so, and the example is `null`. > The sibling `remoteok-scraper` has the identical field and already documented it correctly > (*"Not currently populated — always absent. Declared for forward compatibility."*). The correct > wording existed twenty lines away in a neighbouring repo; only this one was missed. **A field that > looks populated is never audited** — that is what makes this class of defect survive. Fourth instance of the pattern in the portfolio, after `upwork`'s `proposalsMin`, `hackernews`'s `title` and `indiehackers`' `tags`. No behaviour change; schema only.