Glassdoor Company Reviews Scraper
Pricing
Pay per event
Scrape Glassdoor company reviews by company name: rating breakdowns, pros/cons, job title, employment status, review date, and CEO/recommend-to-friend signals, plus a company rating summary. Camoufox-powered to clear the target's bot defenses. Pay only per row scraped.
CEO daily report flagged this listing's Store demo run as failing (evidence:
run bq6vCeZriEHrScB07, build 0.1.7, 2026-09-03 08:08Z —
PROXY_RESIDENTIAL_ TRANSFER_GBYTESrun_usage.py, ruling
out a datacenter/degraded-proxy misread). Checking the live Actor detail
first showed 0.2 (challenge-clear polling + bounded SDK awaits) was already
pushed and live as build 0.2.1 since 2026-09-02 09:26Z — the flagged run
had been pinned to the older 0.1.7 build, not latest.
Three fresh cloud probes against the actual live build (0.2.1, the exact
Store-demo input, 1 company/3 reviews): 1 SUCCEEDED (FgboMQdjOBUPgVYKy, 4
rows), 2 FAILED (sB0pIGOu5Nk1BMshl, dvahR5AXaySh43Jwl) — all three
RESIDENTIAL-confirmed via billing. Reading the FAILED logs found the real bug
0.2's poll fix left in place: the Reviews page's own challenge interstitial
titles itself "Security | Glassdoor", not "Just a moment..." (that title
belongs to the Overview warm-up page's Cloudflare-branded challenge).
_wait_for_challenge_clear's CHALLENGE_TITLE_MARKER was a single-string
constant that only matched "just a moment" — so on the Reviews-page challenge
variant the poll's very first title check already looked "clear" and
returned after one call (title_calls == 1), never actually waiting out the
24s budget the 0.2 fix was built to spend. scraper.py::_is_challenge_page
already carried the full marker list (it correctly logged "reviews-p1 served
a challenge page" in both FAILED runs) — the poll just never shared it.
Fix: browser.py now owns one CHALLENGE_TITLE_MARKERS tuple (5 known
variants) as the single source of truth; _wait_for_challenge_clear checks
against all of them, and scraper.py imports the same tuple instead of
keeping its own copy. New regression test
(test_wait_for_challenge_clear_recognizes_glassdoor_security_title)
reproduces the bug against the pre-fix code (failed with 1 == 3) and
passes after. 92 tests green, ruff clean, pyright clean on every file this
change touches (browser.py/scraper.py/tests/test_browser.py) — three
pre-existing pyright errors in test_models.py/test_rsc_parser.py/
test_scraper.py are unrelated drift, present before this change too, out
of scope here.
Also investigated per the fixer brief's lead: geoip=True (the Camoufox
LeakWarning every run emits). browser.py's own module docstring already
forbids it — a diagnosed 2026-08-06 crash bug (ai-overview-citations) where
geoip=True's IP-echo sweep through the proxy raises Camoufox's own
InvalidIP/InvalidProxy/LocaleError before the browser even launches.
ops/reports/VRBO-GEOIP-CONTRADICTION-2026-08-25.md records that this
rationale was measured on the FREE-tier pool and this account moved to
STARTER on 2026-08-20, so it needs re-proving, not assuming — and per
ops/os/CAP-LIFT-CHECKLIST.md §6b, proving it means testing the launch-
survival question on its own, not bundled with anything else. Live
production evidence from aliexpress-products-scraper (runs
YSBcad4xebfIygDll, RqruyZxtXodLrDEdj, 2026-08-31, same STARTER account,
same RESIDENTIAL pool, geoip_probe=True wired unconditionally in
detail_browser.py) shows Launching Camoufox (... geoip_probe=True) with
no InvalidIP/InvalidProxy/LocaleError crash on either run — the
launch-crash half of the old lesson does not currently reproduce on this
account's tier. That is NOT the same as proving it helps this target: no
cloud run against glassdoor-reviews-scraper's own code with geoip=True set
was possible in this session (deploy is explicitly out of scope — see
notes.md), so it is wired as an opt-in parameter (
open_browser(..., geoip: bool = False)yelp-business- reviews-scrapercountry_code was already pinned to US by default (
ActorInput.country_ codeZero-row exit semantics (_finalize) were also reviewed per the brief: on
zero rows the Actor still raises SystemExit(EXIT_FAILURE) (FAILED), and
EVENT_ACTOR_START still charges before any scrape attempt (unconditional,
same as every PPE Actor in this fleet — see pricing skill's "Actor start...
covers warmup"). Judgment: this is NOT the vrbo/opentable "silently
succeed with 0 rows" bug — Glassdoor's per-company retry loop already
distinguishes a genuine block (AntiBotChallengeError, raised, retried,
FAILS the run when exhausted) from a real empty parse, so REQ-14's
fail-loud-on-zero-rows is the correct call here, not a bug to flip. It is
also NOT the vrbo 100%-blocked pattern that justified delisting: 1/3 fresh
probes SUCCEEDED with real rows on the current live build, consistent with
the ~24-33% probabilistic success rate notes.md already documents for
RESIDENTIAL against this target. Charging the actor-start warmup fee on a
failed attempt is the disclosed, fleet-standard PPE contract, not a
mis-implementation — the lever that actually matters is raising the success
rate (this fix + the geoip experiment above), not changing what gets
charged. Pricing is out of scope for this fixer invocation regardless.
Store demo (companyNames: ["Google"], employerIds: [9079]) was still
failing on live build 0.1.7 (demo_health.py, 2026-09-02) despite 0.1's
AntiBotChallengeError retry. Own-run history stayed FAILED-heavy: 13 of 17
30-day runs failed (fleet_status.py), and qa-ledger.jsonl shows the
latest own probe (run 7I9V8euj7UmHcwxuU, 2026-09-01) FAILED with the
Reviews-page title/html byte-identical to the pre-navigation challenge page.
This lands two fixes that were prepared and locally tested on
fix/glassdoor-reviews-scraper-challenge-poll (2026-09-01) but never
pushed — the branch was intentionally paused pending a fresh evaluation
window for 0.1's fix, and that window is now the demo-health failure above:
src/browser.py::_wait_for_challenge_clear). The flat
WARMUP_SETTLE_MS=3000 wait after navigating to the Reviews page was
too short — ziprecruiter-jobs-scraper's recon (2026-08-31) showed the
same challenge takes 6-18s to self-clear. Now polls up to
CHALLENGE_POLL_ATTEMPTS * CHALLENGE_POLL_INTERVAL_MS (24s) before
reading page content, same pattern as that Actor.Actor.* SDK await (src/main.py) —
fifth fleet instance of the ops/os/UNBOUNDED-AWAIT-TIMEOUTS-2026-09-01.md
pattern. set_status_message/charge now swallow-and-log past a 60s
bound; push_data raises PushStalledError past that bound, caught by
_scrape_one_target so a stall skips only the rest of one company
instead of the whole run. This actor's own failures were logged FAILED
not TIMED-OUT, so this is hardening against the fleet-wide pattern, not
the diagnosed cause of the demo failure — landed together because both
were already tested and reviewed as one unit.Both changes are additive to 0.1's AntiBotChallengeError retry, which
stays in place unchanged. Glassdoor RESIDENTIAL-proxy access remains
probabilistic by nature (docs/specs/glassdoor-reviews-scraper/notes.md)
— this raises the odds a genuinely-clearing challenge is read correctly
instead of read too early, it does not make every run succeed.
glassdoor-salaries-scraper/src/main.py has the identical unwrapped-SDK-
await shape — still flagged, still out of scope for this invocation.
Fix for 33% customer success rate (6/10 30-day FAILED runs). Investigation correction mid-fix, recorded here so the next person doesn't re-walk it:
WEBSHARE_PROXY_URL was removed again; Apify RESIDENTIAL is back to
being the default proxy (unchanged from 0.0 — src/browser.py already
preferred it whenever WebShare is unset).PWError, page.goto succeeds
cleanly — so the existing per-company retry (COMPANY_RETRY_ATTEMPTS,
REQ-9) never triggered; the company was just marked empty on the first
attempt. Live evidence same day: two probe runs back-to-back, one hit
the challenge, the very next (fresh proxy session) SUCCEEDED with 4 real
rows and confirmed PPE charges. Added AntiBotChallengeError (src/ browser.pyRECOVERABLE_BROWSER_ERRORS) plus _is_challenge_page
(src/scraper.py, anchored on the confirmed-absent self.__next_f.push(
wire-format marker, title as a supplementary signal) — a challenge page
now raises instead of silently returning empty, so REQ-9's existing
fresh-session retry actually gets a second exit IP to try.page.content() could race a still-in-flight
client-side redirect on the warm-up page (_read_content_with_retry,
src/browser.py); added _log_page_diagnostics (html length, RSC-chunk
count, marker presence, <title> snippet — never the body) so a future
zero-rows run is diagnosable from the log alone; added the fleet's
missing tests/test_smoke.py (Gate 7 was a silent no-op without it).docs/specs/glassdoor-reviews-scraper/CLOUD-RECON-RESULT.md. Emits
review and company_summary rows. Local test suite green.