Wellfound Startup Jobs Scraper
Pricing
from $1.50 / 1,000 result emitteds
Scrape Wellfound (AngelList Talent) startup job listings by role/keyword and location. Get company funding stage, compensation, remote flag, and tags for every listing — a buying-signal dataset for recruiters and SDRs targeting funded startups.
All notable changes to this Actor are documented here.
zzuPK3zgkC2igPLXx as a delivery
shortfall (maxItems=50, delivered=0). Investigation via apify actor-runs/{id}0.0.2, a recon-only script cited in the 0.1
CHANGELOG entry, never wired to push_data) — not evidence against the
shipped Actor. The shipped Actor (main.py/scraper.py) already had the
real bug, just not the one the flagged run demonstrated: with the
default single-target input (role + location, no searchUrls), a
fully-retried block (WellfoundBlockedError, 5 attempts across rotated
profiles/proxy sessions) degraded to a failed TargetOutcome per REQ-4,
and main() always exited 0 regardless — the comment literally read
"ALWAYS exits 0 here, even when every target failed." A customer whose
only target got blocked was charged actor-start and told the run
SUCCEEDED with 0 rows, indistinguishable from a legitimate empty search.
src/scraper.py now exports DENIAL_REASONS (blocked_retries_exhausted,
request_error, parse_error — reasons meaning the target was never
actually read; no_such_role, a real 404, is excluded because the search
DID complete). src/main.py::_is_denial_failure fails the run
(SystemExit(1)) when zero rows landed AND at least one target failed
for a denial reason — mirroring the upwork-job-leads-scraper 0.3 fix
(positive-marker-required, retry-then-fail-loud). REQ-4's actual intent
is preserved: partial success (some rows landed from any target) still
exits 0, and an all-targets-genuinely-empty run (every target completed,
0 rows) still exits 0 — only the "zero rows because of a block/error/wire
drift" case now fails loud.tests/test_main.py gained unit coverage for _is_denial_failure
(denial-with-zero-rows, genuine-empty, no-such-role, partial-success,
no-outcomes) plus two end-to-end main() regression tests exercising the
full SystemExit path and the still-succeeds-on-genuine-empty path.maxItems
caps exactly, so this Actor is not currently known to be infra-blocked.Un-shelved. The Actor was fully implemented (client.py, parser.py, targets.py, scraper.py — see docs/specs/wellfound-jobs-scraper/) but never published: it carried a shelve note reading
"Cloudflare 403 on all WebShare residential exits"zzuPK3zgkC2igPLXx) swapped ONLY the proxy tier — same
curl-cffi + firefox147 impersonation, same URL shapes, same parser — from
WebShare to Apify's own RESIDENTIAL group pinned country_code=US, with a
fresh exit IP per page (harsher than the original recon's same-session
retry methodology). Result: 20/20 pages HTTP 200, 705 rows parsed, zero
degradation at page 20 vs page 1, circuit breaker never tripped. Apify
RESIDENTIAL was never tried before this — WebShare's residential pool was
reputation-poisoned against this target; Apify's is not.
src/main.py::_build_proxy_factory used to let
WEBSHARE_PROXY_URL (env) silently take precedence over Apify Proxy
whenever the var happened to be set — a dormant trap that could re-route
every future run through the same poisoned pool that produced the false
shelve verdict. The override is gone; Apify Proxy is now the only proxy
path. tests/test_main.py gained a regression test that sets
WEBSHARE_PROXY_URL and asserts it has zero effect on the resolved
proxy URL, alongside the existing no-env-var-set path — both directions
covered.proxy_configuration default now pins apifyProxyGroups: ["RESIDENTIAL"]
apifyProxyCountry: "US"** (src/models.py, .actor/input_schema.json
default + prefill, kept in sync per the prefill-validation gate) — the
exact tier+country the probe confirmed clears the wall, instead of
leaving proxy tier unpinned and hoping a customer's default routing lands
somewhere that works.SHELVED in scripts/publish_actors.py.docs/specs/wellfound-jobs-scraper/ probe notes) —
curl-cffi/RESIDENTIAL has none of the headless-browser egress overhead a
Camoufox path would carry, so PPE pricing at the fleet-standard $0.20
actor-start + $0.0015/result-row (matching recruitee-jobs-scraper /
workday-jobs-scraper) carries a very wide margin.