Amazon Reviews Scraper

A production-grade Amazon Reviews Scraper built for the Apify platform. This scraper extracts high-quality, structured review data from Amazon product pages with advanced filtering, pagination, and GDPR compliance features.

🚀 Features

• Multi-product support: Scrape reviews from multiple Amazon products in a single run
• Smart filtering: Filter by star rating, verified purchases only
• Flexible sorting: Sort by most helpful or most recent reviews
• GDPR compliant: Option to exclude sensitive reviewer data
• Multi-region: Support for 19+ Amazon domains worldwide
• Anti-detection: Browser fingerprinting, proxy rotation, and stealth mode
• Deduplication: Automatic removal of duplicate reviews
• Pagination: Handles all review pages automatically
• Image extraction: Optionally extract reviewer-uploaded images
• Robust error handling: Captcha detection, retry logic, rate limiting

📦 Output Schema

Each review is returned with the following structure:

{
  "productAsin": "B0BDHWDR12",
  "ratingScore": 5,
  "reviewTitle": "Amazing product!",
  "reviewUrl": "https://www.amazon.com/gp/customer-reviews/R1234567890",
  "reviewReaction": "123 people found this helpful",
  "reviewedIn": "Reviewed in the United States on January 15, 2024",
  "reviewDescription": "This is the full review text...",
  "isVerified": true,
  "variant": "Size: Large, Color: Blue",
  "reviewImages": ["https://..."],
  "position": 1,
  "reviewId": "R1234567890",
  "helpfulCount": 123,
  "reviewDate": "2024-01-15T00:00:00",
  "reviewLocation": "United States",
  "sentimentHint": "positive",
  "wordCount": 150,
  "hasImages": true,
  "imageCount": 2,
  "scrapedAt": "2024-01-20T10:30:00Z",
  "sourceUrl": "https://www.amazon.com/dp/B0BDHWDR12"
}

🔧 Input Configuration

Field	Type	Description	Default
productUrls	array	List of Amazon product URLs (required)	-
maxReviews	integer	Maximum reviews per product (1-500)	All available
sortBy	string	Sort order: helpful or recent	helpful
filterByStar	integer	Filter by star rating (1-5)	All stars
verifiedOnly	boolean	Only verified purchases	false
includeImages	boolean	Extract reviewer images	true
includeGdprSensitive	boolean	Include reviewer names/avatars	false
proxyMode	string	automatic or residential	automatic
country	string	Amazon domain	amazon.com
maxConcurrency	integer	Concurrent pages (1-10)	3
requestTimeout	integer	Request timeout in seconds	60
retryCount	integer	Retries for failed requests	3

Example Input

{
  "productUrls": [
    "https://www.amazon.com/dp/B0BDHWDR12",
    "https://www.amazon.com/dp/B08N5WRWNW"
  ],
  "maxReviews": 100,
  "sortBy": "helpful",
  "filterByStar": null,
  "verifiedOnly": false,
  "includeImages": true,
  "includeGdprSensitive": false,
  "proxyMode": "automatic",
  "country": "amazon.com"
}

🌍 Supported Amazon Domains

Domain	Country
amazon.com	United States
amazon.co.uk	United Kingdom
amazon.de	Germany
amazon.fr	France
amazon.it	Italy
amazon.es	Spain
amazon.ca	Canada
amazon.com.au	Australia
amazon.co.jp	Japan
amazon.in	India
amazon.com.br	Brazil
amazon.com.mx	Mexico
amazon.nl	Netherlands
amazon.sg	Singapore
amazon.ae	UAE
amazon.sa	Saudi Arabia
amazon.pl	Poland
amazon.se	Sweden
amazon.com.tr	Turkey

🏗️ Architecture

reviews-scraper/
├── src/
│   ├── main.py              # Apify actor entry point
│   ├── scraper.py           # Core scraping logic
│   ├── url_parser.py        # URL parsing & ASIN extraction
│   ├── review_extractor.py  # HTML parsing & extraction
│   ├── data_normalizer.py   # Data validation & normalization
│   ├── deduplication.py     # Duplicate removal
│   ├── proxy_manager.py     # Proxy & session management
│   ├── cookie_manager.py    # MongoDB cookie rotation
│   ├── cookie_admin.py      # Cookie management CLI
│   ├── email_notifier.py    # Failure email notifications
│   ├── constants.py         # Configuration constants
│   └── utils.py             # Helper functions
├── .actor/
│   └── actor.json           # Apify actor configuration
├── Dockerfile               # Docker build configuration
├── INPUT_SCHEMA.json        # Input schema for Apify
├── requirements.txt         # Python dependencies
└── README.md

🔒 GDPR Compliance

When includeGdprSensitive is set to false (default), the scraper excludes:

• Reviewer names
• Reviewer profile links
• Reviewer avatars

This ensures compliance with privacy regulations while still providing valuable review data.

⚙️ Technical Details

Anti-Detection Measures

1. Browser Fingerprinting: Randomized user agents, viewports, and locales
2. Stealth Mode: Webdriver detection bypass, plugin spoofing
3. Proxy Rotation: Support for datacenter and residential proxies
4. Rate Limiting: Dynamic request throttling
5. Session Management: Automatic session rotation on blocks
6. Human Simulation: Mouse movements, scrolling, and realistic delays

⚠️ Important: Proxies Required

Amazon has aggressive anti-bot detection. For reliable scraping, you MUST use proxies.

• Without proxies: Expect sign-in redirects and captchas
• With datacenter proxies: Works for moderate volume
• With residential proxies: Best success rate

On Apify, enable "Apify Proxy" with residential group for best results:

{
  "proxyMode": "residential"
}

Error Handling

• Captcha Detection: Automatically detects and reports captcha challenges
• Retry Logic: Exponential backoff for failed requests
• Timeout Management: Configurable request timeouts
• Graceful Degradation: Continues with remaining products on failures

Limitations

• Maximum ~100 reviews per star rating (Amazon limitation)
• Maximum ~500 total reviews per product
• Only reviews with text content are extracted

Test Input File

Create storage/key_value_stores/default/INPUT.json:

{
  "productUrls": ["https://www.amazon.com/dp/B0BDHWDR12"],
  "maxReviews": 20,
  "sortBy": "helpful"
}

📊 Example Output

[
  {
    "productAsin": "B0BDHWDR12",
    "ratingScore": 5,
    "reviewTitle": "Best purchase ever!",
    "reviewUrl": "https://www.amazon.com/gp/customer-reviews/R3EXAMPLE123",
    "reviewReaction": "234 people found this helpful",
    "reviewedIn": "Reviewed in the United States on December 15, 2023",
    "reviewDescription": "I've been using this product for 3 months now and it has exceeded all my expectations. The quality is outstanding and the value for money is excellent. Highly recommend to anyone looking for a reliable solution.",
    "isVerified": true,
    "variant": "Size: Medium",
    "reviewImages": [
      "https://images-na.ssl-images-amazon.com/images/I/71example1.jpg"
    ],
    "position": 1,
    "reviewId": "R3EXAMPLE123",
    "helpfulCount": 234,
    "reviewDate": "2023-12-15T00:00:00",
    "reviewLocation": "United States",
    "sentimentHint": "positive",
    "wordCount": 42,
    "hasImages": true,
    "imageCount": 1,
    "scrapedAt": "2024-01-20T10:30:00Z",
    "sourceUrl": "https://www.amazon.com/dp/B0BDHWDR12"
  }
]

🔗 Integration

Export Formats

• JSON: Native output format
• CSV: Export via Apify dataset
• Excel: Export via Apify dataset

API Access

# Get results via API
curl "https://api.apify.com/v2/datasets/{datasetId}/items?token={apiToken}"