Scrape Amazon.com products and Best Sellers lists: give ASINs, product links or Best Sellers category URLs and get price, list price, rating, review count, availability, seller, Best Sellers Rank and categories. Only-changes mode for price monitoring.
Versions follow MAJOR.MINOR.PATCH (src/version.py); Apify shows MAJOR.MINOR from .actor/actor.json.
Every run logs its version and records it in the RUN_STATS key-value record.
0.1.1 (2026-10-16)
A run that returns nothing (every input bad, private, blocked or unreachable, or an empty input) now ends
SUCCEEDED with an empty dataset and the reason as its status message and in the log, instead of FAILED. Only an
unexpected error still fails the run. RUN_STATS keeps the reason as before.
No change to output fields or price.
0.1.0 (unreleased)
First release.
Amazon.com product pages (/dp/<ASIN>, from an ASIN or any product link) and Best Sellers lists
(/gp/bestsellers/<category>[/<node>] or the .../zgbs/... link), up to 100 ranks per list (two pages).
Product: title, brand/byline, price, list price, rating, ratings count, availability and in-stock flag, seller,
shipper, main image, feature bullets, breadcrumb categories, Best Sellers Rank. List row: rank, ASIN, title, price,
rating, ratings count, image for the 30 ranks per page Amazon renders as cards; rank, ASIN and link for the other 20
(the page loads those cards with a signed request we don't make). Optional product details for every list row.
Only new or changed (price, in stock, rank) since the last run with the same inputs, kept in the named store
amazon-products-changes; unchanged rows aren't returned or charged.
A browser User-Agent for www.amazon.com only (Michael's exception to our honest User-Agent, 2026-10-08: Amazon
answered ours with robot checks from Apify's IPs). robots.txt is still read and obeyed, for our product token and
for * both.
Fetching: fixed host www.amazon.com only (addresses are rebuilt from the ASIN or category path, never fetched as
typed), datacenter proxy first, one more try from a residential IP for a refused page, for at most 30% of a run's
pages plus 5. Robot checks (captchas) and 503 block pages are reported as blocked and not charged; captchas are never
solved, no cookies or logins are used.
Charged per result through apify-default-dataset-item; max results per run and the max cost per run are honoured.
Source facts (read 2026-10-08):
robots.txt (https://www.amazon.com/robots.txt ) User-agent: * disallows neither /dp/, /gp/bestsellers/ nor
?pg= paging; it disallows /gp/offer-listing/, /gp/customer-reviews/..., /product-reviews/B0069IY63Y and
others we don't fetch, and disallows everything for AI crawlers (GPTBot, ClaudeBot, CCBot, PerplexityBot ...).
Amazon's Conditions of Use (https://www.amazon.com/gp/help/customer/display.html?nodeId=GLSBYFE9MGKKQXXM , "Last
updated: August 14, 2026"), LICENSE AND ACCESS: the license to use Amazon Services "does not include any resale or
commercial use ...; any collection and use of any product listings, descriptions, or prices; ... or any use of data
mining, robots, or similar data gathering and extraction tools." No page publishes these HTML pages for machines.
Built under Michael's tier-1 decision of 2026-10-08 (PLAN.md "Legal hold lifted for tier 1"); the README quotes
this clause in its "Is it legal" answer.