Amazon Bestsellers Scraper

This changelog summarizes all changes of the Amazon Actors provided by the Junglee organization. The specific actors that are affected are listed for each change.

2024-12-20

Fixes ('amazon-bestseller')

• Fixed extraction of asin on bestseller page after Amazon update

2024-12-18

Fixes ('amazon-reviews')

• Fixed extraction of reviews after today's Amazon update

2024-12-08

Fixes ('amazon-product-pay-per-result')

• Fixed crawling issues related to bad proxies
  • the requests will now be retried, although some timeouts might still occur
• Fixed certain URL parameters being ignored

Fixes ('amazon-product')

• Fixed extraction of product ASIN for specific "half selected" product variants

Features ('amazon-reviews')

• Added support for parsing German, Swedish, and Spanish language dates

2024-12-05

Features ('amazon-reviews')

• Added userId and userProfileLink output properties corresponding to the reviewer's id and profile link

2024-11-26

Fixes ('amazon-product')

• Fixed extraction of prices and "in stock" status for products with "Prime Exclusive Deal" offers

2024-11-24

Fixes ('amazon-product')

• Fixed extraction of prices and "in stock" status for products with "With exchange" offers

Features ('amazon-product', 'amazon-reviews')

• Added the original input product ASIN to the output under the originalAsin field
  • Useful for ASINs that are getting redirected to a different product ASIN

Features (All Amazon Actors)

• Improved handling for non-standard Amazon URLs
  • e.g. on the old http protocol, or with missing www subdomain

2024-11-22

Fixes ('amazon-reviews')

• Fixed language selection (language input option) not working

2024-11-19

Features ('amazon-reviews')

• Added full support for extraction of the .co.jp Amazon domain

Fixes ('amazon-reviews')

• Fixed extraction issues on the .co.jp Amazon domain

2024-11-15

Features ('amazon-reviews')

• Updated scrapeQuickProductReviews to scrape up to 80 reviews at once.
• Also implemented functionality for all review filters such as filterByRating and filterByKeywords etc.

2024-11-10

Features ('amazon-product')

• Added new scrapeProductDetails input option to switch between quick and detailed product scraping
  • (default behavior) - extracts each found product from the category search page in detail
  • (disabled) - extracts only the quick product information from the category search page

2024-11-07

Features ('amazon-reviews')

• Added new variantAttributes field to reviews output
  • Contains the attributes of the product variant in a more structured format (similar to the amazon-product scraper)

Fixes ('amazon-reviews')

• Fixed extraction of variant field values for reviews - the attributes are now correctly split by the , separator
  • Also fixed the extraction for the scrapeQuickProductReviews option

2024-11-05

Fixes ('amazon-reviews')

• Enabled the scrapeQuickProductReviews input option by default
  • This is due to Amazon's recent changes, that have completely hidden all regular reviews behind a login wall

Fixes ('amazon-product')

• Fixed product price extraction for products without a selected variant and the price range being a single value

2024-10-21

Fixes ('amazon-reviews')

• Added temporary fix for the "Most Recent" reviews sort-by method
  • Due to recent changes on Amazon, the "Most Recent" sort-by method is not working - it is hidden behind a login wall
  • You can adjust this option under the new scrapeQuickProductReviews input option
  • It can get up to 8 most recent reviews
• Updated the maximum reviews limit per filter combination from 50 to 100 to reflect Amazon's recent changes

2024-10-13

Fixes ('amazon-reviews')

• Reduced blocking rate

Fixes ('amazon-seller')

• Fixed direct seller URL extraction (e.g. https://www.amazon.com/sp?seller=A2YD2H3KGK1F4L)

2024-09-19

Fixes ('amazon-reviews')

• Updated the maximum reviews limit per filter combination from 100 to 50 to reflect Amazon's recent changes

Changes ('amazon-reviews')

• Removed old functionality for splitting the allStars reviews filter into sub-ratings upon settings max reviews limit to 100+
  • This was confusing and not very useful, as you can split this filter into sub-ratings manually with more control

2024-09-18

Fixes

• Reduced blocking rate ('amazon-reviews')
• Fixed error where the Actor would not start back after a migration (All Amazon Actors)
• Fixed max items limit not working properly for Price-Per-Results Actors ('amazon-bestsellers', 'free-amazon-products')

2024-08-13

Features (All Amazon Actors)

• Improved memory usage, which should resolve some out-of-memory issues

2024-07-29

Features (All Amazon Actors)

• Added proper start URL validation with better handling for invalid URLs

2024-07-22

Fixes (All Amazon Actors)

• Fixed detection for new blocking variation by Amazon

2024-07-16

Features (All Amazon Actors)

• Reduced blocking rate for all Amazon Actors

Fixes ('amazon-bestsellers')

• Fixed pagination and extraction issues for bestseller products Fixes ('amazon-reviews')
• Fixed off by one pagination issue on input

2024-06-13

Fixes ('amazon-product', 'amazon-asins')

• Fixed prices extraction for "buy only used offers" product layout
• Fixed properly truncate offers condition text

2024-06-12

Features ('amazon-product')

• Added new maxOffers input option to scrape product offers under the new offers output field
• Added new scrapeSellers input option to scrape product sellers details under product's sellers field and offers seller field

Fixes ('amazon-product')

• Fixed prices extraction for affected product detail pages

2024-05-23

Fixes (All Amazon Actors)

• Fixed automatic proxy country selection for United Kingdom (amazon.co.uk) Amazon domain

2024-05-23

Features ('amazon-asins')

• Always outputs data, even the product has 0 offers. The product detail data will still be scraped.
  • If no offers are found, adds foundNoOffers: true to the output.

2024-05-21

Fixes ('amazon-product', 'amazon-asins')

• Fixed extraction of product attributes for page layouts with multiple attribute sections

2024-05-20

Features ('amazon-product', 'amazon-reviews', 'amazon-asins')

• Added automatic proxy country selection based on the used Amazon website domain
  • Added under the new proxyCountry input field with a value of AUTO_SELECT_PROXY_COUNTRY

Deprecations ('amazon-product', 'amazon-reviews', 'amazon-asins')

• Deprecated proxyConfiguration input field in favor of the new proxyCountry field

2024-05-17

Fixes ('amazon-bestsellers')

• Fixed extraction issues of blocked prices for bestseller products
• Fixed crawling issues on pages with loaded more products

2024-05-01

Fixes ('amazon-reviews')

• Fixed detection of blocked requests for for non-.com Amazon domains. This caused the scraper to miss some reviews.

2024-04-29

Features ('amazon-reviews')

• Added variantAsin field to reviews output, to distinguish between reviews of different product variants
• Extended the extraction of review date to new languages outside of English
  • Please open an issue for new languages that are not yet supported

2024-04-22

Features ('amazon-product')

• Added support for node search page URLs. Note that these URLs may be invalid if they do not lead to a search page. e.g.:
  • https://www.amazon.com/b?node=4924894011 - valid search page
  • https://www.amazon.com/b?node=468642 - invalid main category page

2024-04-21

Features ('amazon-product', 'amazon-asins', 'amazon-sellers')

• Optimized search page crawling to avoid unnecessary requests

Features ('amazon-reviews')

• Added reviewsCutoffDate input option to singular start URLs
  • Can be changed under the {request}.userData.config.reviewsCutoffDate field

Fixes ('amazon-product', 'amazon-reviews')

• Fixed product ASIN extraction for specific unavailable products

Fixes ('amazon-reviews')

• Fixed incorrect extraction of review date for some languages - null is now returned for these cases

2024-04-15

Fixes ('amazon-product')

• Fixed extraction of product's attributes value fields for specific product page layouts

2024-03-29

Features ('amazon-reviews')

• Added reviews cutoff date filtering option under the new reviewsCutoffDate field
  • Filters out only reviews that were posted since the specified date

Fixes ('amazon-product')

• Fixed relative URLs extraction for seller.url and reviewsLink fields to absolute URLs

2024-03-18

Features ('amazon-product')

• Added extraction for price ranges for products with selection options under the new priceRange field

2024-03-11

Features ('amazon-product')

• Added extraction for "AI reviews summary" under the new aiReviewSummary field
• Added extraction of sponsored product status for search page results under the new categoryPageData.isSponsored field

Features (All Amazon Actors)

• Added language selection for all Actors under the new language input field

2024-02-21

Features ('amazon-reviews')

• Implemented reviews deduplication to avoid scraping the same reviews for overlapping review filters and keywords

2024-02-20

Features ('amazon-reviews')

• Added scrapeProductDetails input field to scrape product details for the inputted product ASINs
  • If enabled, scrapes the same data as the amazon-product actor does, but also scrapes the reviews.

2024-02-15

Features ('amazon-reviews')

• Added filterByKeywords input field. Can be used to get more relevant reviews or to get more reviews than the default 100 reviews limit.
  • e.g. keywords that usually get lots of results: ["Good", "Bad", "Great", "Quality", "Useful", "Love", "Price"]

2024-02-14

Features (All Amazon Actors)

• Improved our previous attempts on decreasing Amazon's high blocking. (especially for amazon-reviews)

2024-02-04

Features ('amazon-reviews')

• Added reviewsUseProductVariantFilter and reviewsEnqueueProductVariants input fields to overcome the reviews limit
  • reviewsUseProductVariantFilter allows you to filter reviews for a specific product variant
  • reviewsEnqueueProductVariants allows you to crawl and scrape reviews for all product variants automatically

Fixes ('amazon-product')

• Fixed variantAsins not being properly extracted for all product variants
• Fixed variantDetails extraction not working for products with 'size' variants and products with carrier subscriptions

2024-02-02

Features (All Amazon Actors)

• Significantly decreased the frequency of Amazon's recently elevated blocking rate. (especially for amazon-reviews)

Features ('amazon-product')

• Added extraction of Amazon's A+ content under aPlusContent field
  • content like text, images and videos is scraped under rawText, rawImages, and rawVideos fields
  • specific A+ content modules are scraped fully under the modules field

2024-01-06

Fixes ('amazon-product')

• Fixed extraction of Amazon Fresh products, when crawled from a product category page.

2024-01-04

Features ('amazon-product')

• Added option to scrape prices for all variants of a product under variantDetails field
  • You can toggle this option under the scrapeProductVariantPrices input field
  • Note: this feature will increase the number of requests and extend the scraping time

2023-12-25

Features (All Amazon Actors)

• Significantly decreased the frequency of Amazon's recently elevated blocking rate.
  • We've made major changes to our backend, if you encounter any issues, please reach out to us. Thank you!

2023-12-19

Features ('amazon-product', 'amazon-sellers')

• Added further support for Amazon category pages in languages other than English (language is set by your Amazon domain - .fr, .es, .com.br etc.)

2023-12-13

Features ('amazon-product', 'amazon-asins', 'amazon-sellers', 'amazon-bestsellers')

• Amazon has pushed an update that removes product prices as a form of anti-scraping blocking. We've added a detection for this and the Actor will now retry with a different proxy if it detects this issue.

2023-11-18

Features ('amazon-reviews')

• Updated filterByRating to filterByRating*s* - now works as a multi-select field (e.g. ["fiveStar", "fourStar"])
  • The original filterByRating field still works the same way. You can still access it through the JSON input editor.

Reverted ('amazon-product')

• Removed detailedInformation input field. Shouldn't affect any existing tasks. It was just removing these fields:
  • attributes, productOverview, and manufacturerAttributes
• Added the useCaptchaSolver input field back. We've noticed some slight errors with it, you can turn it off if you experience any issues.
  • Amazon hides a few certain product fields, if the page loads after solving a captcha. Currently known and affected fields are:
    • attributes, manufacturerAttributes, and bestsellerRanks

2023-11-09

Features ('amazon-product', 'amazon-sellers', 'amazon-bestsellers')

• Added maxItemsPerStartUrl input field to limit the number of items to be scraped per each start URL
  • The original maxItems field still works the same way. You can still access it through the JSON input editor.

('amazon-reviews')

• Added product.price & product.listPrice output fields to reviews
  • Note: these fields are tied to the review's product, not the review. We can add more product fields to the output but we don't want to bloat it with too many fields. If you need more product fields, please open an issue about it ;)

('amazon-product')

• Added force country codes feature - you can select the country codes only for which you want to scrape the product. The Actor will retry with a different proxy if a page is loaded with the wrong country code.
  • Note: this feature is experimental and is only available through the JSON input editor under the forceCountryCodes field (string[]).

2023-11-05

Fixes ('amazon-reviews')

• Fixed extraction of reviewer's user avatar URL
• Fixed max reviews limit not working properly (Amazon lowered the max review count per each review page)

2023-11-04

Fixes

• Fixed issue with Amazon captcha solver not working properly

2023-10-23

Fixes ('amazon-sellers')

• Fixed pinned product offer price extraction

2023-10-22

Fixes

• Fixed "Received unknown page layout" error for Gift Card product pages

2023-10-18

Fixes

• Fixed the order of product's bestseller ranks in bestsellerRanks field.

2023-10-17

Features

• Added book description extraction for products in the Books category under bookDescription field

2023-10-14

Features

• Added tracking of product positions within their respective category/listing pages under categoryPageData field
  • categoryPageData: { productPosition: number, categoryUrl: string }

2023-10-09

Features

• Added productOverview field with main product attributes

2023-10-07

Fixes ('amazon-sellers')

• Fixed certain seller pages not being parsed correctly

2023-09-25

Features

• Added price to variantDetails field

2023-09-10

Features ('amazon-sellers')

• Add support for different seller URL format: https://www.amazon.com/sp?s=XXXXX

2023-08-04

Features ('amazon-reviews')

• new output field reviewCategoryUrl
  • holds URL of a category the review was extracted from (e.g. 4 star reviews category)
• total numbers of ratings and reviews are injected into each review
  • totalCategoryRatings field holds the number of ratings for a given category
  • totalCategoryReviews field holds the number of reviews for a given category

2023-08-01

Fixes ('amazon-reviews')

• maxReviews limit is applied per each product separately

2023-07-12

Fixes ('amazon-reviews')

• Better handling of maxReviews limit
• Fetching 20 reviews per XHR page instead of 10

2023-07-10

Features ('amazon-reviews')

• Rating filters
  • Reviews by stars, positive or critical reviews
  • filterByRating input field

2023-06-29

Features

• Added more product images: galleryThumbnails and highResolutionImages
• Added variantDetails with variant name, thumbnail and images