Free-plan runs returned 0 records. Every worker's proxy session was labelled amz0-0, amz1-0…, and Apify Proxy accepts only letters, digits, _, . and ~ in a session id — so building the proxy URL itself failed with ValueError, before any request left the container. The log showed it as "search page 1 failed", which read like Amazon refusing the page; runs on our premium gateway were unaffected because that gateway does not validate the label, which is why only free users hit it.
- Session ids are now
amz<run>_<worker>_<n> — valid for Apify Proxy, and with a random per-run prefix, so a resurrected or repeated run no longer lands on the sticky IPs the previous run already used.
- A proxy URL that cannot be built is now reported as what it is — an Actor bug — and fails the run at start-up, instead of being swallowed page by page as if Amazon had failed.
- Verified live on the free path (Apify Residential) and on the paying path. Known separate issue: amazon.fr currently answers its home and search pages with an AWS WAF JavaScript challenge (HTTP 202) to every client, so
.fr search returns nothing for now; product pages and charts on .fr are not affected.
A run was only as fast as the one household it happened to rent, and it paid for every page Amazon refused to serve. Both came from the same place: the whole run went out through a single proxy address, and the gate Amazon puts in front of a new address was only recognised in English.
- Every worker now has its own address. The proxy session id was asked for without a lane, so all four workers of a run shared one sticky address and queued behind one home connection. A customer run of 70 products on amazon.de took 24 m 59 s — 21.4 s per product with four workers — while the same code on a normal address does 1.5 s. Each worker (
maxConcurrency) now holds its own address, its own cookie jar and its own delivery address, and rotates only its own on a refusal. Measured on the same 12 products, back to back: 34.8 s → 17.8 s (2.9 → 1.5 s per product; 70 products ≈ 1.8 minutes).
- The "continue shopping" gate was English-only, and every miss cost a full page twice. Amazon answers a cold address with a 4 KB interstitial — "Weiter shoppen" on
.de, "Continuer mes achats" on .fr — and the Actor looked for the English wording. Anywhere but .com the gate was therefore read as a soft block: the page was thrown away, a new address was drawn, and it met the same gate. Measured 21.09.2026 on amazon.de: 2 of 12 products died with all six attempts spent. The gate is now recognised by its form (a GET to /errors_page/validateCaptcha with the amzn / amzn-r fields, identical in every language), cleared on the address that got it, and the page is re-requested there — 2 KB instead of a second copy of a 2.5 MB page.
deliveryZipCode no longer costs a second copy of a product page per worker. The address-change token lives in the header of any Amazon page, so the run now takes it from Today's Deals (339 KB) before the worker's first product, instead of taking it from the product page and downloading that page again for the corrected prices. Five cold addresses, each page tried on all five: Today's Deals 5/5 with a token at 339 KB, Best Sellers 5/5 at 533 KB, the cart 0/5 (Amazon answers a new address with 503), a product page 2.5–3 MB. If the cheap page is refused anyway, the old path still runs, so the first record of a worker is never priced for the proxy's city.- The premium gateway is reached over HTTP now, not SOCKS5. Measured from an Apify container, four requests in parallel: over SOCKS5 one or two arrived and the rest died with "curl (97) Failed to receive SOCKS response, proxy closed connection"; over HTTP, four out of four. Every one of those deaths cost an address change, and the fresh address met the gate and downloaded the page again — which is why the same code did 1.5 s per product from a laptop and 23 s on the platform.
- Connections and cookies live for the worker now, instead of a fresh TLS session with an empty cookie jar for every single page. Fewer handshakes, and Amazon sees one browser instead of a new stranger on every request — which is what the gate reacts to.
- The run was starved of CPU, and that was most of the 25 minutes. Apify hands out processor time in proportion to a run's memory, and this Actor asked for 512 MB — an eighth of a core — while parsing 1.5–3 MB of HTML per product. Measured in the container: a page downloads in 0.7 s but took 14–19 s from request to parsed record, with the processor pinned and the network idle. A run now asks for 2 GB and cannot be given less. The same 70 products: 24 m 59 s at 512 MB, 6 m 38 s at 2 GB, 3 m 8 s at 4 GB — for the same platform cost (0.211, 0.221 and 0.210 compute units), because the work is the same work, just no longer queued behind a sliver of a core. 2 GB is the default rather than 4 because the container's start-up is billed per run whatever its size, and at 4 GB that start-up alone outweighs a one-product run; anyone scraping thousands can raise the memory and get the rest of the speed for free.
- Traffic, end to end: 27.8 MB of HTML for 12 products → 23.2 MB, with the records identical field for field (the only two differences between the runs were a coupon that appeared and a list price that disappeared on Amazon's side in the meantime).
- The cost line in the log stopped overstating. Wire volume is estimated from the HTML volume, and one ratio (3.4×) was used for every page although the measurement behind it said 3.4× for listings and 5.4× for full pages. Re-measured on four live amazon.de product pages: 8.6 MB of markup compress to 1.63 MB (5.3×). Product runs were being reported at about 1.5× their real proxy cost; each kind now uses its own measured ratio.
The Actor could only read English. Every field Amazon prints as words was matched against the English wording, so on any other marketplace it came back empty — or silently wrong. A run of 70 ASINs on amazon.de returned stars: null on all 70 products and rating: null on all 508 reviews, while ratingDistribution (which is numbers, not words) was perfect. This release reads those fields by structure instead of by English, and every one of the 19 marketplaces was re-verified end to end in its own language.
- Ratings. "4,3 von 5 Sternen", "4,5 sur 5 étoiles", "5つ星のうち4,3", "5 üzerinden 4,5 yıldız" — the rating is now read as the pair number + the 5 of the scale, in either order, which needs no list of languages. It fixes
stars on products, search cards, category and chart cards, deal cards and variants, rating on every review, and sellerRatingStars on offers.
- Review counts on listing pages were multiplied by a billion. The abbreviation suffix ("34.7K", "34,7 Tsd.") was matched without a word boundary, so the first letter of the next word became the multiplier: German "1.307 Bewertungen" came out as 1.307 B = 1,307,000,000 and "760 Bewertungen" as 760 billion. Product records were unaffected — the product page prints the count with no word after it, which is why the same run had correct counts in
product mode and nonsense in search.
- Sponsored cards were never flagged.
isSponsored looked for the word "Sponsored" and for a label class on a <span>; Amazon writes the label in the language of the store ("Gesponserte Anzeige", "Annonce sponsorisée") and hangs that class on an <a>. The flag now comes from what does not change between stores: the ad-label class and the /sspa/click redirect link. Two consequences of the same bug are fixed with it — the ad label ("Gesponserte Anzeige – ") no longer leaks into title, and a sponsored card whose link is javascript:void(0) no longer puts that string into url, which now falls back to /dp/<ASIN>.
deliveryZipCode finally does something. The address change was signed with the token printed on the page (glowValidationToken); Amazon does not accept it there, and answers 200 with an empty body — so the Actor believed the address was set, or (since 0.6) reported that Amazon "kept its own location". The working token is the one Amazon prints inside the address dialog, which the Actor now opens first (25 KB, once per run). Verified live: 10115 → Berlin 10115 on amazon.de, 10001 → New York 10001 on .com, and on .co.uk, .it, .es, .se, .pl, .ca, .com.mx, .com.br, .co.jp, .in, .com.au, .sg and .com.tr. Two stores — amazon.nl and amazon.sa — have no postcode field in that dialog at all, only a country; a postcode sent there is refused in every format, so the run now falls back to the store's own country and says so in the log, instead of silently keeping the prices of whatever country the proxy landed in.- Prices came in the proxy's currency. amazon.de billed a US exit node in dollars —
priceText: "8,17USD" — and the page even prices differently for a foreign address (€7.11 for a US visitor, €6.95 for Berlin). The Actor now asks each store for its own currency, so a .de run is in EUR whatever address the proxy pool hands out. Two parsing fixes come with it: a currency printed after the number ("8,17USD", "7,95 €", "315,74 kr") is read instead of dropped, and a unit price in the same string ("16,99 € (0,17 €/ml)") no longer merges into the number.
currency is now always an ISO code — EUR, not sometimes EUR and sometimes €; USD on .com where it used to be $. The symbol as printed is still in priceText.- Stock, review origin and review dates stopped being English-only:
inStock reads "Auf Lager" / "Derzeit nicht verfügbar" and their equivalents, country / location on a review parse "Bewertet in Deutschland am …", "Comentado en México el …", "2024年7月5日に日本でレビュー済み", and dateIso now understands Portuguese, Polish, Turkish, Swedish, Dutch and Arabic month names (and Arabic-Indic digits).
- Cookies were kept under
.com names only. The cookies that carry a passed bot-check and the chosen address are named per store (ubid-main on .com, ubid-acbde on .de), and only the .com spelling was remembered — so on every other marketplace both were dropped after the first request. They are now matched by prefix.
The run pays for itself now. The platform bills this Actor for what it downloads through the proxy, and that line was 85–90 % of every run's cost (measured on the usage breakdown: $0.0041–0.0048 of proxy traffic against $0.0004–0.0012 of compute). Amazon serves HTML in gzip only — it answers a request for br or zstd with an uncompressed body — so there is nothing to win on compression, and every saving had to come from downloading less. This release stops paying for four things that were never used:
- The page tail is no longer downloaded. Past the last block the parser reads, an Amazon page continues with the footer and ~50 JS bundles. The body is now read as a stream and the connection is dropped at the end of the useful markup: a
/s? listing goes from 278 KB to 231 KB over the wire (−17 %), a product page from 482 KB to 453 KB (−6 %), measured with a byte counter on the socket. Listings get cut earlier still — at the filter panel, which the Actor never parses (rh= tokens come from its own table) — but only once result cards have already arrived, because one of the two layouts Amazon rotates emits that panel before the results.
- The session check no longer costs a page. Verifying the logged-in session used to fetch
/gp/css/homepage.html on every run — a full Amazon page for zero rows of data, and up to nine of them when the account pool handed out a signed-out session. The verdict now comes off the first page the run downloads anyway; the header that carries it is the same on every Amazon page. Dead accounts are still flagged back to the pool, and still only after two different egress IPs refuse the session.
deliveryZipCode no longer re-downloads every page. Changing the address is a POST plus a reload, and the reload is a second full copy of the page. It was repeated on every single request, because each request opened a fresh session with an empty cookie jar. The address cookies are now kept for the run, so the change happens once. And when Amazon ignores the change — it answers the POST with 200 and keeps its own location, which is what it does on .com today — the run says so in the log instead of paying for that reload on every page.offers no longer treats "no offers" as a block. Amazon's All-Offers endpoint answers an ASIN with no sellers with a legitimate 4.8 KB fragment. The Actor read the size, called it a soft block, retried six times, then downloaded the whole ~1.9 MB product page to scrape a buy-box line that had no seller on it. A valid AOD answer is now recognised by its own container, an empty one is reported as empty, and the product-page fallback reuses the page the run already has instead of fetching it again. One ASIN in that state went from 6 requests / 1.8 MB to a single 5 KB fragment.
Fixed: the two "also fetch…" toggles were free. includeOffers and includeSellerDetails are documented — in the input form itself — as charging one offer event per offer and one seller event per product, and they never did. Each of them is a separate Amazon page, so a product record with both cost three pages of traffic and billed for one cheap event. They now charge what the form says they charge. Reviews embedded in a product record stay free: those arrive inside the same HTML and cost nothing extra, which was always the reason they were free.
Cheaper waiting. The retry back-off no longer grows to 9.8 s (1.5 → 2.4 → 3.8 → 4 → 4 instead), because compute units are billed for every second the container is up, and past the fourth attempt the pause is not what decides the outcome. The product parser also stopped reading the same two values three times each — the rating and the review count were parsed once for the log, once for the record and once for hasReviews.
Paid plans now go out through our own network. Requests for paying Apify users leave through the developer's premium gateway instead of Apify Residential; free users are unchanged. There is nothing to configure and nothing to bring — the Actor has never had a proxy input and still doesn't. The gateway is used in session-sticky mode, one stable address per proxy session: Amazon issues its bm-verify token and its "continue shopping" form to a specific address, so a network that rotated per request would arrive at the gate as a stranger on every step. The address rotates only when a request actually fails, exactly as before. A gateway that cannot reach Amazon answers with a short text body rather than an error, and that body is now recognised and retried from another address instead of being parsed as a page — on a pay-per-event Actor, parsing it would have meant charging for garbage. All twelve modes were re-verified end to end over the new path (12/12, 17 requests, no captcha and no retries).
New in the log: what the run cost. Every run now ends with the number of pages, the HTML volume, the wire volume behind it and the dollar figure, plus how many pages were cut short. Traffic is the thing that decides whether a mode earns or loses, and it used to be invisible.
Six new scrape modes. charts (Best Sellers / New Releases / Movers & Shakers / Most Wished For / Gift Ideas by category slug), storefront (every product a seller lists), deals (Today's Deals, no input needed), stores (Amazon /stores/ brand pages), asinLookup (EAN / UPC / ISBN → ASIN) and reviewById. All six bill the existing searchResult / review events — no new prices.
Product records grew from ~40 to 55+ fields, including the ones competitors sell as extras:
monthlyPurchaseVolume / monthlyPurchaseCount — Amazon's "10K+ bought in past month", verbatim and as a number. The demand signal that does not lag the way review counts do.variantDetails — every sibling variant with its name, attributes and thumbnail, instead of a bare ASIN list.frequentlyBoughtTogether, productComparison, aPlusContent, brandStory, brandStoreUrl, galleryThumbnails, productOverview, importantInformation, sustainabilityFeatures + climatePledgeFriendly, isAmazonChoice, priceRange, condition, offersCount, ratingDistribution, authors / bookDescription, hasReviews / reviewsLink, originalAsin / unNormalizedProductUrl.includeOffers and includeSellerDetails attach the full offer list and the buy-box seller's profile to the product record — one run instead of three.
Reviews embedded in another record are now free. A product record carries the reviews Amazon prints on the page and a seller record carries its feedback rows; neither is billed on top any more. A product used to cost its own event plus one review event per embedded review — up to 6× the headline price for data that arrived in the same HTML.
Delivery address. New deliveryZipCode takes a ZIP, postcode or two-letter country code and fetches every page against it, so prices, stock and delivery dates stop drifting with the proxy's city.
Search filters. searchMinPrice, searchMaxPrice, searchBrand, searchSellerId, searchPrimeOnly, searchFourStarsAndUp, searchCondition and searchDeals. Price and brand work on every marketplace; the rest use Amazon's per-marketplace node IDs and are skipped with a log line where the ID is not known, rather than silently returning an empty result.
Review filters and fields. reviewsDateFrom / reviewsDateTo / reviewsFilterByKeyword, plus dateIso (the raw "Reviewed in … on …" string stays in date), userId, variantAsin, variantAttributes, position, totalCategoryRatings and totalCategoryReviews.
Seller records gained email, storefrontUrl, productAsins and the published shippingPolicy / returnPolicy / privacyPolicy / helpContent / otherPolicies.
Known limit, now stated instead of silently empty: of the five charts, Amazon serves four as ready-made HTML; Movers & Shakers is drawn in the browser and its pages contain no product markup at all. That chart returns nothing and now says why in the log, in the input schema and in the README.
Fixed: the run threw away every IP that had just been let through. Apify's proxy pins an IP to a session id, and the Actor asked for a new one on every single request. Amazon's gates are per-IP — the bm-verify token and the "continue shopping" form clear the address they were issued to — so each request arrived at the gate as a stranger and no run ever got past it on a flagged pool. The proxy session is now held for the whole run and rotated only after a request actually fails, and the cookies Amazon hands out on the way through are reused by every later request instead of being dropped with the session.
Fixed: runs returned empty rows instead of saying they were blocked. Amazon answers a flagged IP with a 3 KB "Click the button below to continue shopping" page or a 150-byte stub, both HTTP 200. These parsed into zero cards and the log said "0 reviews" for a product with 57,000 of them. Stub pages are now detected, the "continue shopping" gate is followed automatically like the bm-verify interstitial already was, and a genuinely blocked page is reported as blocked. JSON responses from Amazon's ajax endpoints are exempt from the size check.
Fixed: seller mode could return an almost-empty record. A bare /sp?seller=… request gets served Amazon's internal products JSON instead of the profile page often enough to matter; the request now carries the full parameter set Amazon's own "Sold by" link uses.
Fixed: listPriceText came back as the string "null" on listings without a struck-through price, and deliveryLocation came back as the button label "Update location" when no address was set. Both are omitted now.
Cheaper runs. Default memory is 512 MB instead of 1 GB (the Actor is async HTTP plus BeautifulSoup — compute units scale with memory). Retries went from 10 fixed 2-second attempts to 6 with growing pauses: every failed attempt downloads the full page before the block is recognised, so ten of them cost several times what the record sells for. The reviews fallback no longer stacks three rounds of retries on top of the six that already ran underneath.
- Fixed: review counts on search, category and Best-Sellers rows were up to 1000× too small. Amazon now abbreviates the count it prints next to a card ("34.7K"), and the Actor read the digits only: a listing with 34,720 ratings came back as
reviewsCount: 347, and a Best-Sellers row with 279,601 ratings as 44. The exact number is taken from the rating link Amazon keeps beside it ("34,720 ratings"); where only the abbreviation exists it is expanded properly (K/M/B and their localised twins, so amazon.de's "34,7 Tsd." works too).
- Fixed:
isPrime was false on every search row. The a-icon-prime badge no longer exists in Amazon's current delivery block, so Prime eligibility is now read from the delivery line itself — Prime rows are flagged again, and rows that are not eligible ("FREE delivery Sat, Sep 5 on $35 of items shipped by Amazon") stay false.
- Search rows no longer disappear when Amazon changes the card layout. A second card layout is in rotation (signed-in runs get it) and a row whose
<h2> was missing used to be dropped from the dataset without a trace. The title is now read from every slot a layout can put it in, so the row is emitted either way — and where a layout offers both a shortened and a full title, the full one wins.
- Product-page counts (reviews, answered questions, helpful votes) and a seller's rating count in the offers list go through the same abbreviation-proof parser instead of silently returning a wrong number or none at all.
- Amazon put a password on its review pages.
/product-reviews/{ASIN} and the individual review pages now answer with a sign-in form even for a session that is demonstrably logged in everywhere else, and Amazon's lazy-widgets review stream returns an empty body. Reviews mode therefore returns the 8–13 reviews printed on the product page — the same set signed in or not. The ~32 signed-in reviews of 0.2 are no longer reachable by anyone; product mode's customersSay / reviewAspects remain the way to read the full corpus in aggregate.
- Reviews mode no longer wastes requests on the wall. The wall is now recognised (its
/ap/signin link sits 54 KB deep, past the window the old check looked at), so a run stops after it instead of retrying review pages, individual review pages, and the lazy-widgets stream in turn: 2 requests per ASIN instead of up to 9, without re-downloading the ~1.8 MB product page twice.
- Fixed: a signed-in run could return 3 reviews where an anonymous one returned 13. The product-page reviews parsed during warm-up were kept only when
/product-reviews/ had also returned cards, so once the wall went up they were dropped and the run fell back to whatever the deep review pages happened to give. They are now always emitted, deduplicated by review ID.
- Signing in still works and is still on by default — it affects delivery dates today, and review pages are picked up automatically if Amazon reopens them.
- Offers mode now returns every seller, not just the buy-box. Amazon moved its All-Offers endpoint, so the Actor had been silently falling back to the single buy-box offer for every ASIN; a typical listing now returns 3–6 offers with condition, seller ID, FBA flag and delivery. No login needed.
- ~2.5× more reviews on signed-in runs (~32 vs ~13 on a typical ASIN): the Actor now also collects the reviews Amazon references from its own review analysis but never lists.
- New in product mode:
customersSay + reviewAspects. Amazon's own AI summary of the whole review corpus, plus per-aspect sentiment (Sound quality — positive, 1.7K mentions) with a paragraph per aspect. This is the signal from the tens of thousands of reviews Amazon no longer serves individually.
- Signed-in runs. New
useOurAccounts toggle (on by default) borrows a logged-in Amazon session from our own account pool for the run and returns it afterwards; amazonCookies accepts your own EditThisCookie export and takes priority over the pool. Both are optional — without them the run works exactly as before, anonymously.
- Reviews: sources are now merged instead of first-wins. The signed-in
/product-reviews/ page and the reviews embedded on the product page are combined and deduplicated by review ID, so a signed-in run never returns fewer reviews than an anonymous one.
- Fixed: review pages parsed as empty. Review cards are
<li data-hook="review"> on /product-reviews/ but <div> on the product page; the tag-specific selector silently found zero cards on every review page.
- Fixed:
reviewTitle contained "5.0 out of 5 stars" instead of the review headline on signed-in review pages.
- Fixed:
variant contained "Verified Purchase" — the badge sits inside the same block as the size/colour strip and is now subtracted.
- Fixed:
isVine was true for every verified-purchase review (the selector matched the generic success-colour class, not the Vine badge).
- Fixed: star / media / verified review filters returned unfiltered results. Amazon now ignores
filterByStar & co. in the URL and serves the same review set regardless, so the Actor applies the filters itself — asking for 1-star reviews no longer returns 5-star ones.
- Fixed: buy-box
sellerName was "Learn more about the seller" on signed-in runs; the seller name is now read from the merchant block and the link is only used for the seller ID and URL.
- Review pagination stops as soon as a page repeats the previous one — Amazon now serves page 1's reviews for every page number.
- Fixed: a working account could be dropped from the pool after a single signed-out page. Amazon serves an anonymous page to live sessions too — from an unlucky proxy IP, or on a marketplace the cookies were not exported from — and the Actor retired the account on the first one. The verdict now needs two different IPs to agree, and a session that another marketplace refuses is returned to the pool instead of being flagged. Your own
amazonCookies now say so explicitly when they belong to a different marketplace than the run.
- Product detail scraping: title, ASIN, price, list price, currency, stars + rating breakdown, reviews count, breadcrumbs, features, A+ description, hi-res images, videos, variants, BSR, seller, delivery, return policy, tech-spec table.
- Search / category / Best-Sellers listing rows: ASIN, title, price, stars, reviews count, Sponsored / Prime / Best-Seller / Amazon's Choice flags, badge, coupon, delivery, position, page.
- Customer reviews: rating, title, body, date, country, verified-purchase flag, helpful votes, variant, photos, videos, linked to parent ASIN.
- Seller profiles: business name, address, VAT, trade register, phone, registration country, overall stars, total feedback, Lifetime / 12-month / 90-day / 30-day positive % with 5-star histograms.
- Buy-box / AOD offers: price, list price, currency, condition, seller name+ID+URL+rating, Prime, Fulfilled-by-Amazon, delivery, shipping price.
- 19 Amazon marketplaces (US, UK, DE, FR, IT, ES, JP, IN, AU and more). No Amazon API, no captcha. Pay-per-event pricing.
