Amazon Product Scraper

What is Amazon Product Scraper and How Does It Work?

Amazon Product Scraper is a powerful web scraping tool that extracts product data from Amazon using product URLs. It collects product details like price, stock availability, ratings, reviews, seller info, and more — all exportable in JSON or CSV format.

Why Use Amazon Product Scraper?

Use this scraper to:

• Monitor pricing and stock for competitor products
• Track product ratings and reviews for market analysis
• Benchmark product performance within categories
• Automate data extraction for E-Commerce analytics and advertising optimization

How to Use Amazon Product Scraper

1. Enter one or more Amazon product URLs.
2. Set the maximum number of products URLs to scrape.
3. Run the scraper and download the extracted data.
4. Optionally, access the data via API without logging into Apify.

Features

• Scrapes product details from Amazon product URLs
• Supports both single URL and multiple URLs as input
• Extracts comprehensive product data:
  • Title
  • Price
  • Images (all product images)
  • Variants and their prices
  • Ratings (average rating)
  • Reviews (configurable number of reviews)
  • Product details and specifications
  • Product description

Usage

Input Configuration

The actor accepts the following input parameters:

{
  "urls": [
    "https://www.amazon.com/product-url-1",
    "https://www.amazon.com/product-url-2"
  ],
  "maxReviews": 10
}

Running on Apify

To run this scraper on Apify, follow these steps:

1. Create an Apify account if you don't have one already at apify.com

2. Create a new actor in your Apify Console:

   • Go to Apify Console
   • Click on "Actors" in the left sidebar
   • Click "Create new" button
   • Choose "Custom actor"

3. Upload your code:

   • You can either upload a ZIP file containing all the files in this repository
   • Or connect your GitHub repository if you've pushed this code there

4. Build the actor:

   • Once your code is uploaded, click the "Build" button
   • This will build a Docker image based on your Dockerfile

5. Run the actor:

   • Click the "Run" button
   • Input your configuration (URLs and maxReviews)
   • Start the run

6. Access the results:

   • Once the run is complete, you can access the scraped data in the "Dataset" tab
   • You can download the data in various formats (JSON, CSV, Excel, etc.)

Running Locally with Apify CLI

You can also run this actor locally using Apify CLI:

1. Install Apify CLI: npm install -g apify-cli
2. Navigate to the project directory
3. Run apify run to start the actor locally
4. Access the results in the apify_storage directory

• urls (required): One or more Amazon product URLs to scrape. Can be a single URL string or an array of URLs.
• maxReviews (optional): Maximum number of reviews to scrape per product (default: 10, max: 100)

Example Input

{
  "urls": [
    "https://www.amazon.com/dp/B08N5KWB9H",
    "https://www.amazon.com/dp/B08N5LNQCX"
  ],
  "maxReviews": 20
}

Output

The actor outputs a dataset with detailed information about each product. Each item in the dataset contains:

• url: The URL of the product page
• title: Product title
• price: Current product price
• images: Array of product image URLs
• details: Object containing product specifications and details
• description: Product description
• average_rating: Average product rating (out of 5)
• review_count: Total number of reviews
• variants: Array of product variants with their names and prices
• reviews: Array of product reviews, each containing:
  • title: Review title
  • rating: Individual review rating
  • date: Review date
  • text: Review content
  • reviewer: Reviewer name

Local Development

Installation

1. Clone this repository
2. Install the required dependencies:

   pip install -r requirements.txt
   

3. Install Playwright browsers:

   playwright install
   

Running Locally

To run the actor locally:

python main.py

You can provide input by creating a file named INPUT.json in the same directory with your input configuration.

Deployment to Apify

To deploy this actor to the Apify platform:

1. Install the Apify CLI:

   npm -g install apify-cli
   

2. Log in to your Apify account:

   apify login
   

3. Deploy the actor:

   apify push
   

Anti-blocking Measures

This actor uses several techniques to avoid being blocked by Amazon:

• Random user agent rotation
• Headless browser with realistic behavior
• Proper request throttling and timing

Limitations

• The actor may not work for all Amazon regional domains
• Some product details may vary depending on the product category
• Amazon's website structure changes frequently, which may require updates to the scraping logic

License

This project is licensed under the MIT License - see the LICENSE file for details.