Amazon Reviews Scraper
Pricing
Pay per event
Scrape the top customer reviews from an Amazon product page by ASIN or URL — rating, title, body, verified-purchase flag, date, and helpful-vote count per review. Pay-Per-Event, no login, no API key.
Pricing
Pay per event
Rating
0.0
(0)
Developer
DevilScrapes
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
3 days ago
Last modified
Categories
Share
{FAILED: 6, SUCCEEDED: 1}, 14%
success, failures still accumulating as of the 08-03/08-05 snapshots):
compared two cloud QA runs from build 0.2.1 that already ran the
0.1/0.2 fixes — wCnILYz9DB0d7e4MK (FAILED, statusMessage: "Scraped 0 reviews across 1 ASIN(s): B0BSHF7WHW (no-results)"TiDTHYXnIXMOPwZGI (SUCCEEDED, 5 real review rows) — same ASIN, same
build, three minutes apart. The failing run's log shows exactly one
soft block (HTTP 200) — rotating session line, then a real,
non-block 200 response that is_bot_block_page() correctly does
not flag (it has a productTitle) but that embeds zero
data-hook="review" blocks — a real Amazon page, just missing the
Top-reviews widget. The succeeding run needed 3 rotations before
landing on a working exit and returned 5 rows for the identical
ASIN. The only thing that varies run-to-run is which residential exit
gets minted; per today's cross-actor finding (bestbuy.com
country-selector splash, Viator JPY pricing), residential proxy
exits are geo-random, and amazon.com renders a different,
reviews-less layout for a non-US-recognized visitor without ever
tripping a bot-check marker. The actor never pinned an exit country,
so ~1 in a few rotations silently produced this false-empty result
instead of erroring.src/main.py::_resolve_proxy_configuration: now calls
Actor.create_proxy_configuration(groups=..., country_code="US")
instead of passing the raw proxyConfiguration input straight
through — every request is pinned to a US residential exit
(PROXY_COUNTRY_CODE), and RESIDENTIAL is force-included
(_ensure_residential) even if a caller's proxyConfiguration input
omits it. marketplace_domain is a single-value "amazon.com" enum
in v1, so hardcoding US is in-scope, not a workaround.tests/test_main.py: two new tests pin that
_resolve_proxy_configuration always passes country_code="US" and
always includes "RESIDENTIAL" in groups, regardless of what the
input's proxyConfiguration dict contains.test_proxy_session_id_matches_apify_regex,
Gate 9's regression test), ruff clean, pyright clean,
scripts/verify_input_prefill.py re-verified./publish-actor's Gate 1 and Gate 7): added
.actor/icon.png (rendered from the existing icon.svg via
qlmanage, matching scripts/upload_actor_icons.py's fallback
path) and tests/test_smoke.py (opt-in -m smoke, live fetch of
ASIN B0BSHF7WHW's product page, asserts parseable reviews) — this
actor had neither before, unlike the google-ads-transparency
reference implementation.tests/conftest.py's
autouse _patch_sleep fixture called
monkeypatch.setattr(client_mod.asyncio, "sleep", ...) —
client_mod.asyncio is the real, global asyncio module object (a
Python import doesn't copy it), so this silently replaced
asyncio.sleep process-wide for every test, not just calls made
from src.client. That starves curl-cffi's async transport (which
depends on genuine event-loop scheduling to service a live socket),
turning a real network fetch into an indefinite CPU-spinning hang
instead of a clean timeout — reproduced directly: the first draft of
test_smoke.py hung >3 minutes at ~60% CPU with this patch active,
and passed in under 2s once the fixture was scoped to skip
smoke-marked tests. Fixed via request.node.get_closest_marker ("smoke")test_client.py
still pass unaffected (they're not smoke-marked).recent_failures at build
0.1.1 (vvSBwVsHWFjaF44YX, PImgd1oHcBK0rI5pT, both started
2026-08-01 after the 0.1 soft-block fix was already live) via
apify runs info — both failed in 6-11s with statusMessage: "Invalid input: 1 validation error for ActorInput. Value error, Provide at least one of asins or product_urls."INPUT KVS record showed every
field at its input_schema.json default — asins: [],
productUrls: [] — not the prefill value (["B0BSHF7WHW"]).
prefill only backs the interactive Console form; a run started
without customizing every field (bare API call, integration,
platform health-check) gets the schema's default, and
asins.default had drifted to [] while asins.prefill stayed a
real ASIN — so any such invocation hard-failed REQ-3 immediately.
This is a distinct bug from the 0.1 soft-block fix (confirmed via
fresh log pulls, not assumed) and was invisible to
scripts/verify_input_prefill.py's existing Gate 3c because that
check only reconstructs the prefill payload, never default..actor/input_schema.json: asins.default now matches
asins.prefill (["B0BSHF7WHW"]) instead of [] — a bare/default
invocation now runs a real (small, demo) scrape instead of failing
loud on empty input. ActorInput's REQ-3 cross-field validator is
unchanged: a customer who explicitly clears both fields and submits
still gets a clear validation error, as intended.tests/test_models.py: new test_schema_default_input_is_runnable
reconstructs the exact payload Apify merges for an un-customized
run (every property's schema default, mirroring the real
vvSBwVsHWFjaF44YX/PImgd1oHcBK0rI5pT INPUT records) and pins that
it must pass ActorInput.model_validate — regression guard for this
default/prefill drift class, since Gate 3c alone doesn't catch it.verify_input_prefill.py
re-verified against the updated schema.apify-actor-start/apify-default-dataset-item events, not this
repo's custom pay_per_event.json events — main.py's
Actor.charge() calls remain no-ops on the live platform pending
reconciliation.apify api
cloud runs against build 0.0.1 — of 3 fresh runs on the same ASIN,
one FAILED with statusMessage: "Scraped 0 reviews across 1 ASIN(s): B0BSHF7WHW (no-results)"marketing/opportunities/SCRAPER-BACKLOG-2026-07-29.md item #1)
intermittently soft-blocks the Actor's proxy exit IPs: it answers
with HTTP 200, not a 403/429, so client.py's existing retry/rotate
logic — which only branches on status code — never engaged. The
blocked page has no productTitle and no data-hook="review"
blocks, so parse_review_page correctly found zero reviews and
main.py correctly failed a run with a genuinely empty result — the
bug is that a soft block was silently treated as "this product has
no reviews" instead of "retry with a fresh session", exactly as
client.py's original module docstring called out as scraper.py's
responsibility but which was never wired up.src/parser.py: adds is_bot_block_page() — detects Amazon's
bot-check interstitial via known text markers
(BOT_BLOCK_TEXT_MARKERS, industry-documented, not recon-captured
this session — flagged for re-verification against a live capture)
plus a structural fallback (no product title AND no review blocks;
a real product page always has a title, even with zero reviews).src/client.py: AmazonReviewSession.get_page() now accepts an
optional is_blocked predicate; a 200 response it flags rotates the
proxy session + browser profile and retries, identical to the
existing 403/429 path, instead of being returned as real content.src/scraper.py: wires parser.is_bot_block_page into
session.get_page(...).src/models.py, .actor/input_schema.json: proxyConfiguration
default/prefill now includes apifyProxyGroups: ["RESIDENTIAL"] —
Amazon is an anti-bot-prone target per
DEVILSCRAPES-CONTEXT.md's anti-blocking-stack rule; the actor
previously defaulted to Apify's datacenter proxy group.tests/test_client.py (soft-block rotate+retry,
exhausted-retries FetchError, backward-compat no-predicate path),
tests/test_parser.py (is_bot_block_page true/false cases incl.
a real zero-review product staying False), tests/test_scraper.py
(regression pin that is_bot_block_page is actually wired into
get_page).amazon.com/product-reviews/<ASIN> (the
originally-planned all-reviews listing page) 200s but silently
redirects to amazon.com/ap/signin — it is login-walled, not
anti-bot-walled (no CAPTCHA/JS-execution challenge; confirmed via
r.url after redirect and inspecting the sign-in HTML). No
curl-cffi impersonation profile change fixes an auth gate.amazon.com/dp/<ASIN> (the product page) is not login-walled —
200, no redirect, and it embeds a real, keyless "Top reviews from
the United States" widget (data-hook="review" blocks; 8 observed
on a live sample fetch, data-hook="total-review-count" showing
~196k total ratings the widget does not paginate through over plain
HTTP).amazon.com/dp/<ASIN> (typically 6-10 per ASIN), not a full
paginated review corpus. This is a narrower but real, live-confirmed
surface — see docs/specs/amazon-reviews-scraper/spec.md "Wire
format" for the full pivot rationale.src/parser.py: selectors are now live-confirmed (not placeholders)
against a real fetch; fixed TITLE_CSS (h5[data-hook="reviewTitle"],
not a[data-hook="review-title"]) and BODY_CSS
(div[data-hook="reviewRichContentContainer"], not
span[data-hook="review-body"]) to match real markup. All other
selectors matched the original hypothesis exactly.src/scraper.py: removed the /product-reviews/ pagination loop
(sortBy/pageNumber, MAX_PAGES_PER_ASIN); now a single fetch of
amazon.com/dp/<ASIN> per ASIN. Dedupe (REQ-10) kept as a defensive
guard against duplicate markup on one page.src/models.py: dropped sort_by/sortBy (only meaningful for the
removed pagination); max_reviews_per_product ceiling lowered from
2000 (default 100) to 20 (default 10) to reflect the real, smaller
surface — no longer misleading about corpus size..actor/input_schema.json, README.md: updated to describe "top
reviews from the product page" honestly instead of a full,
paginated corpus; dropped the sortBy input field.tests/fixtures/review_page_page1.html /
review_page_empty.html now mirror the real /dp/<ASIN> markup
structure confirmed via live recon (data-hooks, tag names, nesting)
with anonymized review text/names — not blind synthetic guesses.verify_input_prefill.py re-verified
against the updated input_schema.json..actor/ (actor.json, input/output/dataset schemas, PPE
events, Dockerfile, icon), src/__init__.py + src/__main__.py
entry point, tests/conftest.py + tests/fixtures/input.qa.json,
README skeleton.src/models.py, src/client.py, src/parser.py, src/scraper.py,
and src/main.py are not implemented yet — see
docs/specs/amazon-reviews-scraper/tasks.md (T02-T11).src/recon_cloud.py, orchestrator-owned,
never shipped) has not yet confirmed the wire-format hypothesis in
docs/specs/amazon-reviews-scraper/spec.md ("Wire format") — parser
selectors are placeholders pending that recon.