EBay Scraper Ultimate

eBay Scraper Ultimate is a sophisticated and robust Apify Actor designed to extract comprehensive product data from eBay. It goes beyond simple title and price extraction, providing deep insights into product variants, ratings, and customer feedback.

Features

• Anti-Bot Measures: Implements randomized delays (2-5 seconds), human-like behavior (random mouse movements, slow incremental scrolling), and optimized concurrency to bypass eBay's sophisticated detection.
• Robust Title & Data Extraction: Uses multiple fallback selectors, increased timeouts, and page.evaluate for reliable data capture across various eBay page layouts, including the new "evo" layout.
• Bot Detection Handling: Automatically detects and handles eBay's "Pardon Our Interruption" screens, logging warnings and skipping blocked pages to ensure crawl continuity.
• Enhanced Image Gallery: Extracts all available product images, including high-resolution versions (up to 1600px) from the gallery, with automatic image cleaning.
• Deep Item Specifications: Captures comprehensive technical details and item specifics using advanced table-parsing logic and text-based search fallbacks.
• Seller Insights: Extracts seller names and feedback ratings to help identify reputable sources.
• Reliable Ratings & Reviews:
  • Average Rating: Extracts the average star rating using reliable LD+JSON structured data.
  • Rating Count: Captures the total number of ratings for the product.
• Customer Feedback: Scrapes the top 10 most relevant customer comments/reviews for each product.
• Residential Proxy Support: Defaults to Apify Residential Proxies for maximum reliability and anti-bot protection on the Apify platform. Gracefully falls back to no proxy for local development.
• Optimized for Performance: Combines Playwright for dynamic content and BeautifulSoup for fast, resilient data extraction.

How to Use

Input Parameters

The Actor accepts the following input parameters:

Field	Type	Description	Default
searchQuery	String	Required. The product name or keywords to search for on eBay.	iPhone 15
maxProducts	Integer	Maximum number of products to scrape.	20
proxyConfiguration	Object	Apify Proxy or external proxy configuration for rotating IP addresses.	{ "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"] }

Proxy Configuration

The Actor defaults to Apify Residential Proxies for best results and anti-bot protection.

On Apify Platform

When running on the Apify platform with an account that has proxy access enabled:

• Residential Proxies (Default): Automatically enabled for maximum reliability and bot detection avoidance
• Datacenter Proxies: Can be configured by changing apifyProxyGroups to ["DATACENTER"]
• External Proxies: Select "Custom proxies" and provide your own proxy URLs (e.g., http://username:password@proxy.example.com:8080)

Local Development

When running locally with apify run:

• The Actor will display a warning and run without proxies
• This is normal behavior - residential proxies only work on the Apify platform
• The scraper will still function but may be more prone to blocks for large-scale scraping

[!IMPORTANT] Residential proxies are highly recommended for production scraping to prevent bot detection and ensure high reliability. They are automatically enabled when running on the Apify platform.

Output Data

The results are stored in the default dataset. Each item in the dataset represents a product and contains the following fields:

• url: The direct link to the eBay product page.
• title: The full title of the product.
• price: The current listed price.
• condition: The condition of the item (e.g., "New", "Very Good - Refurbished").
• variants: An object containing available options (e.g., {"Color": ["Black", "Blue"], "Storage": ["128GB", "256GB"]}).
• averageRating: The average star rating of the product.
• ratingCount: The total number of ratings received.
• sellerName: The name of the seller.
• sellerRating: The feedback rating of the seller.
• itemDetails: An object containing detailed specifications of the item.
• images: An array of product image URLs.
• comments: An array of the top 10 customer comments.

Local Development

If you want to run this Actor locally, ensure you have the Apify CLI installed.

1. Install dependencies:

   $pip install -r requirements.txt
   

2. Install Playwright browsers:

   $playwright install chromium
   

3. Run the Actor:

   $apify run
   

[!NOTE] When running locally, you'll see a warning about proxy access. This is expected - the Actor will continue without proxies. For production use with proxies, deploy to the Apify platform.

Deployment to Apify

To deploy this Actor to the Apify platform:

1. Log in to Apify:

   $apify login
   

2. Push the project:

   $apify push
   

Resources

• Apify SDK for Python
• Crawlee for Python
• Beautiful Soup Documentation
• Apify Proxy Documentation