Amazon Category Scraper

Scrapes products from Amazon category and search result pages. Extracts detailed product data including prices, reviews, seller info, variants, and delivery details. Supports multiple Amazon domains.

Features

• Scrapes Amazon search results and category pages
• Extracts detailed product information from individual product pages
• Supports pagination (up to 50 pages per category)
• Handles multiple Amazon domains (US, UK, DE, FR, etc.)
• Anti-detection measures (user agent rotation, stealth scripts)
• CAPTCHA detection and simple bypass attempts
• Proxy support with automatic country selection

Input

Field	Type	Description	Default
categoryUrls	array	Required. List of Amazon category or search URLs	-
maxItemsPerCategory	integer	Maximum products to scrape per category (1-1000)	100
maxPages	integer	Maximum pages to scrape per category (1-50)	10
scrapeProductDetails	boolean	Scrape full details from product pages	true
proxyCountry	string	Proxy country (AUTO, US, GB, DE, etc.)	AUTO
rateLimitDelay	integer	Delay between requests in seconds (1-30)	2

Example Input

{
  "categoryUrls": [
    "https://www.amazon.com/s?k=laptops",
    "https://www.amazon.com/s?k=wireless+headphones"
  ],
  "maxItemsPerCategory": 100,
  "maxPages": 10,
  "scrapeProductDetails": true,
  "proxyCountry": "AUTO",
  "rateLimitDelay": 2
}

Output

Each product is saved as a separate item in the dataset with the following fields:

Field	Type	Description
title	string	Product title
url	string	Product URL
asin	string	Amazon Standard Identification Number
inStock	boolean	Whether product is in stock
inStockText	string	Raw availability text
brand	string	Product brand
price	object	Current price {value, currency}
listPrice	object	Original/list price {value, currency}
shippingPrice	number	Shipping cost (null if free)
stars	number	Average star rating (0-5)
starsBreakdown	object	Rating breakdown {5star, 4star, ...}
reviewsCount	integer	Total number of reviews
answeredQuestions	integer	Number of answered questions
breadCrumbs	string	Category breadcrumbs
thumbnailImage	string	Main product image URL
description	string	Product description
features	array	Product feature bullet points
variantAsins	array	ASINs of product variants
reviewsLink	string	Link to reviews page
delivery	string	Primary delivery estimate
fastestDelivery	string	Fastest delivery option
returnPolicy	string	Return policy text
support	string	Support information
variantAttributes	array	Variant attribute names
priceVariants	object	Price by variant
seller	object	Seller info {name, id, url, ...}
bestsellerRanks	array	Bestseller rankings
locationText	string	Delivery location text
categoryUrl	string	Source category URL
categoryPage	integer	Page number in category
categoryPosition	integer	Position in category results
scrapedAt	string	Timestamp of scraping

Example Output

{
  "title": "SanDisk 1TB Extreme microSDXC UHS-I Memory Card",
  "url": "https://www.amazon.com/dp/B09X7MPX8L",
  "asin": "B09X7MPX8L",
  "inStock": true,
  "inStockText": "In Stock",
  "brand": "SanDisk",
  "price": {
    "value": 145.5,
    "currency": "$"
  },
  "listPrice": {
    "value": 299.99,
    "currency": "$"
  },
  "shippingPrice": null,
  "stars": 4.8,
  "starsBreakdown": {
    "5star": 0.86,
    "4star": 0.09,
    "3star": 0.02,
    "2star": 0.01,
    "1star": 0.01
  },
  "reviewsCount": 36704,
  "answeredQuestions": 151,
  "breadCrumbs": "Electronics > Computers & Accessories > Memory Cards",
  "thumbnailImage": "https://m.media-amazon.com/images/I/716kSUlHouL.jpg",
  "description": "Save time with card offload speeds...",
  "features": [
    "Up to 190MB/s read speeds",
    "Up to 130MB/s write speeds",
    "4K and 5K UHD-ready"
  ],
  "variantAsins": ["B09X7C2GBC", "B09X7C7LL1"],
  "reviewsLink": "/product-reviews/B09X7MPX8L",
  "delivery": "Thursday, January 26",
  "fastestDelivery": "Sunday, January 22",
  "returnPolicy": "Eligible for Return within 30 days",
  "support": "Free Amazon tech support included",
  "variantAttributes": [],
  "priceVariants": null,
  "seller": {
    "name": "Direct Suppliers US",
    "id": "A210SJF12S88M5",
    "url": "/gp/help/seller/at-a-glance.html",
    "reviewsCount": null,
    "averageRating": null
  },
  "bestsellerRanks": [
    {"rank": 3, "category": "Micro SD Cards"}
  ],
  "locationText": "Deliver to New York",
  "categoryUrl": "https://www.amazon.com/s?k=memory+cards",
  "categoryPage": 1,
  "categoryPosition": 5,
  "scrapedAt": "2026-01-26T10:30:00.000Z"
}

Supported Amazon Domains

• amazon.com (US)
• amazon.co.uk (UK)
• amazon.de (Germany)
• amazon.fr (France)
• amazon.it (Italy)
• amazon.es (Spain)
• amazon.ca (Canada)
• amazon.com.au (Australia)
• amazon.co.jp (Japan)
• amazon.in (India)
• amazon.com.br (Brazil)
• amazon.com.mx (Mexico)
• And more...

Usage Tips

1. Fast Mode: Set scrapeProductDetails: false to only extract data visible on category pages. This is much faster but provides less detail.

2. Rate Limiting: Increase rateLimitDelay if you encounter CAPTCHAs or blocks.

3. Proxy: The scraper automatically selects the appropriate proxy country based on the Amazon domain.

4. Pagination: Amazon typically shows 20-60 products per page. Set maxPages accordingly.

Local Development

# Install dependencies
pip install -r requirements.txt

# Install Playwright browsers
playwright install chromium

# Run the scraper
python -m src

License

This project is licensed under the MIT License.