eBay Scraper — Apify Actor

A full-featured Apify actor that scrapes eBay search results and individual product pages. Built in Python with async support, anti-detection measures, and automatic layout detection for eBay's 2025/2026 page structures.

---

Features

• 3 scraping modes — search, product, or both (search → deep-scrape each result)
• Auto layout detection — handles eBay's 2026 card layout (li.s-card) and legacy layout (li.s-item) with fallback selectors
• Deep product extraction — title, price, original price, condition, seller info, feedback score, item specifics (Brand, MPN, etc.), high-res images, shipping, returns, delivery estimate, quantity sold, categories, and JSON-LD structured data
• Filtering & deduplication — filter by price range and condition; auto-deduplicate by item ID
• 8 eBay domains — US, UK, DE, FR, IT, ES, CA, AU
• 5 sort options — best match, price low/high, ending soon, newly listed
• Anti-detection — Sec-Fetch headers, randomized delays, Apify residential proxy support, CAPTCHA detection with graceful stop
• Single-file architecture — everything in one main.py for easy deployment

---

Quick Start

1. Deploy to Apify

# Clone and push
apify init
apify push

Or paste main.py directly into the Apify Console code editor.

2. Run Locally

pip install apify httpx beautifulsoup4 lxml
apify run --input input.json

3. Example Inputs

Search mode — scrape search listings:

{
    "mode": "search",
    "searchQueries": ["led headlight bulbs", "mechanical keyboard"],
    "maxPages": 3,
    "sortBy": "price_low",
    "minPrice": 10,
    "maxPrice": 100,
    "conditionFilter": "new",
    "ebayDomain": "ebay.com"
}

Product mode — scrape specific listings:

{
    "mode": "product",
    "productUrls": [
        "https://www.ebay.com/itm/177800312518",
        "https://www.ebay.com/itm/123456789012"
    ]
}

Both mode — search then deep-scrape every result:

{
    "mode": "both",
    "searchQueries": ["sony wh-1000xm5"],
    "maxPages": 2,
    "maxItems": 20
}

---

Input Reference

Parameter	Type	Default	Description
mode	string	"search"	search, product, or both
searchQueries	string[]	[]	Keywords to search on eBay
productUrls	string[]	[]	Direct eBay item URLs to scrape
maxPages	int	3	Max search result pages per query (1–20)
maxItems	int	0	Stop after N items per query (0 = no limit)
minPrice	float	0	Min price filter in dollars (0 = off)
maxPrice	float	0	Max price filter in dollars (0 = off)
conditionFilter	string	"any"	any, new, used, or refurbished
sortBy	string	"best_match"	best_match, price_low, price_high, ending_soon, newly_listed
ebayDomain	string	"ebay.com"	Target eBay domain (e.g. ebay.co.uk, ebay.de)
proxyConfig	object	Apify residential	Proxy configuration
delayMin	float	4	Min random delay between requests (seconds)
delayMax	float	9	Max random delay between requests (seconds)

---

Output Schema

Search Results

Each item in search mode outputs:

{
    "searchQuery": "led headlight bulbs",
    "itemId": "177800312518",
    "title": "LED Headlight Bulbs 6000K White H11 Low Beam",
    "price": "$24.99",
    "priceNumeric": 24.99,
    "condition": "Brand New",
    "shipping": "Free shipping",
    "imageUrl": "https://i.ebayimg.com/images/g/...",
    "link": "https://www.ebay.com/itm/177800312518",
    "scrapedAt": "2026-03-16T12:00:00+00:00"
}

Product Details

Each item in product / both mode outputs:

{
    "url": "https://www.ebay.com/itm/177800312518",
    "itemId": "177800312518",
    "title": "LED Headlight Bulbs 6000K White H11 Low Beam",
    "price": "$24.99",
    "priceNumeric": 24.99,
    "originalPrice": "$49.99",
    "originalPriceNumeric": 49.99,
    "currency": "USD",
    "condition": "Brand New",
    "quantityAvailable": 342,
    "quantitySold": "1287",
    "shipping": "FREE Expedited Shipping",
    "deliveryEstimate": "Estimated between Wed, Mar 18 and Fri, Mar 20",
    "sellerLocation": "Shenzhen, China",
    "returns": "30 days returns. Buyer pays for return shipping.",
    "seller": {
        "name": "auto_lights_direct",
        "feedbackScore": "45,231",
        "positiveFeedback": "98.7%",
        "topRated": true
    },
    "itemSpecifics": {
        "Brand": "Auxbeam",
        "Bulb Type": "LED",
        "Color Temperature": "6000K",
        "Wattage": "50W",
        "Fitment Type": "Direct Replacement"
    },
    "images": [
        "https://i.ebayimg.com/images/g/.../s-l1600.jpg"
    ],
    "categories": ["eBay Motors", "Parts & Accessories", "Car & Truck Parts"],
    "scrapedAt": "2026-03-16T12:00:00+00:00"
}

---

Project Files

ebay-apify-actor/
├── main.py              ← All code (scraper, helpers, filters, actor logic)
├── input_schema.json    ← Apify input UI schema
├── requirements.txt     ← Python dependencies
├── Dockerfile           ← Apify build config
└── .actor/
    └── actor.json       ← Apify actor metadata

---

How It Works

Layout Detection

eBay frequently changes its HTML structure. The scraper auto-detects the page layout:

Layout	Detected By	Era
2026 (card)	li.s-card elements	Current
Legacy (s-item)	li.s-item elements	Pre-2025

Each layout has its own set of CSS selectors for title, price, link, condition, image, and shipping. If eBay changes again, just add a new selector set to SEARCH_SELECTORS.

Anti-Detection

The scraper uses several techniques to avoid blocks:

1. Browser-like headers — Sec-Fetch-*, Accept-Language, Upgrade-Insecure-Requests
2. Random delays — configurable range (default 4–9 seconds) between requests
3. Apify residential proxies — rotates IP per request
4. CAPTCHA detection — detects "Pardon Our Interruption" pages and stops gracefully

Product Page Extraction

The product scraper uses a multi-strategy approach:

1. CSS selectors — multiple fallback selectors per field (modern → legacy)
2. JSON-LD — extracts structured Product data from <script type="application/ld+json">
3. Table parsing — reads item specifics from both flex layouts and <table> elements
4. Image resolution upgrade — rewrites s-l300 thumbnails to s-l1600 for full resolution

---

Customization

Adding a New eBay Domain

Add the domain string to input_schema.json's ebayDomain enum:

"enum": ["ebay.com", "ebay.co.uk", "ebay.de", "ebay.co.jp"]

No code changes needed — the domain is passed directly to URL construction.

Adding New Selectors

If eBay rolls out a new layout, add an entry to SEARCH_SELECTORS:

SEARCH_SELECTORS["2027"] = {
    "container": "div.new-card-class",
    "title": [".new-title-selector"],
    "price": [".new-price-selector"],
    ...
}

Then update _detect_layout() to recognize the new container.

Custom Post-Processing

Use the apply_filters() and deduplicate() functions, or add your own:

# Example: keep only items with images
items = [i for i in items if i.get("imageUrl") != "N/A"]

---

Troubleshooting

Problem	Solution
CAPTCHA blocks	Use residential proxies (RESIDENTIAL group); increase delayMin/delayMax
0 items found	eBay may have changed layout — inspect page HTML and update selectors
Missing prices	Some auction items show bids instead of prices; check price field for "bid" text
Timeout errors	Increase timeout in fetch_page() or reduce maxPages
Empty product fields	eBay loads some content via JavaScript — CSS selectors can only read server-rendered HTML

---

Dependencies

Package	Purpose
apify	Actor framework, dataset, proxy management
httpx	Async HTTP client with proxy support
beautifulsoup4	HTML parsing
lxml	Fast HTML parser backend

---

License

MIT — use freely for personal and commercial projects.