eBay Scraper

A Python 3.12 Apify Actor that uses a warm, browser-held Camoufox session to collect public eBay search listings and item-page details. It supports active-listing search and direct item URLs across eight marketplaces.

What it does

• Searches ebay.com, ebay.co.uk, ebay.de, ebay.fr, ebay.it, ebay.es, ebay.ca, and ebay.com.au.
• Parses current li.s-card and .su-card-container markup plus legacy li.s-item cards.
• Supports active search and detail (item URL) modes.
• Warms the marketplace homepage before every held-session search and retries a blocked unit with a fresh browser session (at most three attempts).
• Applies a strict maxResultsPerQuery cap to each query.
• Optionally opens each result page for JSON-LD/DOM enrichment.
• Deduplicates cards by eBay item ID within each query.
• Writes flat listing rows to the default dataset and aggregate analytics/failures to the OUTPUT key-value-store record.

Input

At least one searchQueries or itemUrls value is required. detail mode requires an item URL.

• searchQueries: keyword strings.
• itemUrls: direct eBay item URLs. These always produce truthful detail rows, even alongside a search mode.
• mode: active or detail.
• marketplace: one of the eight domains above.
• maxResultsPerQuery: 1–1000, strict per-query cap.
• maxPagesPerQuery: 1–20.
• minPrice, maxPrice: optional non-negative search filters.
• condition: any, new, used, refurbished, or for-parts.
• listingType: all, auction, or buy-it-now.
• sortBy: best-match, price-lowest, price-highest, newly-listed, or ending-soonest.
• scrapeItemDetails: enrich search records from item pages, including a listing start date only when an explicit machine-readable date is present.
• includeSalesEstimates: defaults to true. Adds the estimate fields described below. Set to false to preserve listing rows without any estimator fields.
• maxConcurrency: 1–20.
• proxyConfiguration: defaults to Apify Residential proxies. Availability and billing depend on your Apify plan.

Example:

{
  "searchQueries": ["vintage mechanical watch"],
  "mode": "active",
  "marketplace": "ebay.com",
  "maxResultsPerQuery": 100,
  "maxPagesPerQuery": 3,
  "condition": "used",
  "scrapeItemDetails": false,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}

Dataset fields and nulls

Rows include item ID, canonical URL, title, status, numeric price and original price text, currency, shipping, condition, listing type/bids, seller fields, quantity sold, image fields, item specifics, query, marketplace, and scrape timestamp. Detail rows also expose quantityAvailable only when an exact non-negative integer is present in a machine-readable offer inventory field; unit-count prose is never treated as availability or sales. Search cards expose different fields by layout and market. A field is null when eBay did not visibly provide it; the Actor does not manufacture values.

price and currency reflect the currency eBay served the request in, which follows the buyer's location and is not always the marketplace's own currency. When a listing shows an amount in a currency the Actor cannot identify, currency is null rather than being labelled with the marketplace default.

Sold and completed listings are not available

eBay gates sold/completed search (LH_Sold) behind sign-in for signed-out users, returning a redirect to signin.ebay.com carrying its own sgfl=srch flag. This applies to eBay's own "Sold Items" filter link, on desktop and mobile web, and on non-US marketplaces from matching local IPs. It is an authorization gate, not a bot challenge: the same session serves active search normally. This Actor therefore does not offer a sold mode and does not estimate transaction prices from asking prices.

No synthetic sales history is generated. quantitySold remains the observed cumulative listing count shown by eBay, read from the listing's own quantity element; it is never relabeled as monthly or recent sales, and a recency badge such as "6 sold today" is never used as that count.

Standalone sales estimates

The runtime is fully standalone: it uses no database, history service, model server, or additional API. No validated trained model artifact currently ships with this Actor. Therefore the current deterministic estimate is available only when eBay exposes reliable cumulative quantitySold, detail HTML has an explicit machine-readable listing date (used to calculate nonnegative listingAgeDays against scrapedAt), and the listing is not an auction or known single-unit listing.

The fallback is quantitySold / listingAgeDays * 30. It is labeled salesEstimateMethod: "lifetime_average", has low confidence, and is a derived lifetime average—not observed recent sales. Vague page prose is never used to infer age. Missing or ineligible inputs produce null estimate/interval fields and insufficient_data. Auctions and known single-unit listings do not receive unbounded unit estimates without a separately validated probability model.

estimatedMonthlyRevenue exists only when estimate and item price exist. It is monthly units × item price and excludes shipping, tax, fees, discounts, and conversion. Basis, provenance, confidence, method, and model version fields make this explicit.

{
  "quantitySold": 60,
  "listingAgeDays": 120,
  "price": 20.0,
  "estimatedMonthlySales": 15.0,
  "estimatedMonthlySalesLower": null,
  "estimatedMonthlySalesUpper": null,
  "estimatedMonthlyRevenue": 300.0,
  "salesEstimateMethod": "lifetime_average",
  "salesEstimateConfidence": "low",
  "salesEstimateModelVersion": null,
  "salesEstimateProvenance": "Derived from observed cumulative quantitySold and explicit listing age; not recent observed sales.",
  "salesEstimateBasis": "Lifetime-average monthly units (quantitySold / listingAgeDays * 30); revenue uses item price only (shipping excluded)."
}

A future artifact must declare validated: true, artifact and feature-contract versions, model version and monthly count target, log-link intercept, allowlisted runtime-only numeric/categorical predictors, coefficients, and uncertainty calibration. Invalid, future-contract, target/admin-feature, or runtime-incompatible artifacts are rejected and safely fall back. searchRank may be an optional model feature, but it is eBay query rank—not Amazon BSR—and has no standalone sales meaning.

Offline training

python -m pip install -r requirements-training.txt
python -m training.build_labels --help
python -m training.build_labels snapshots.jsonl labels.jsonl --horizon-days 30 --tolerance-days 3 --rejected-output rejected.jsonl
python -m training.train_model --help
python -m training.train_model labels.jsonl --holdout-fraction 0.2 --max-wape 0.75 --max-absolute-bias 0.30

The label builder groups timestamped snapshots by stable item/variation, sorts them, and uses forward cumulative-sold deltas near the horizon. It rejects resets, missing/censored/ended/sold/unavailable records, auctions, known single-unit listings, variation or timestamp ambiguity, and unavailable horizons. Features come only from the earlier snapshot to prevent leakage.

The trainer uses an interpretable scikit-learn Poisson/log-link model and a final temporal holdout. Before fitting, it embargoes every training row whose label window extends beyond the earliest holdout start and purges matching item IDs, relist chains, and product groups. Production defaults require at least 500 total, 400 post-purge training, and 100 holdout rows, including at least 20 zero-sale and 50 nonzero-sale holdout examples. Tests may explicitly lower those gates; artifacts record both configured and production minimums, observed counts, and whether production minimums were used. The trainer reports MAE, WAPE, RMSLE, aggregate bias, zero-sales diagnostics, and empirical uncertainty multipliers. Validation exports nothing by default. JSON is written only with explicit --export PATH and passing WAPE/bias gates. Synthetic tests establish tool correctness, not real-market accuracy; representative real labels are required before bundling an artifact.

OUTPUT analytics

OUTPUT reports emitted row counts, explicit failures, and per-query price analytics: count, min, max, average, median, and the p10/p25/p75/p90 percentiles, plus a condition mix.

These are asking prices on active listings at scrape time, not transaction prices, and every row carries that statement in priceBasis. A listing's asking price is what a seller wants, not what anything sold for. Percentiles are computed only over rows sharing the query's dominant currency; anything in another currency is excluded and counted in priceOtherCurrencyExcludedCount. Ranking, pagination, duplicate suppression, and the configured caps all shape which listings are sampled.

Blocking and operational behavior

eBay changes markup and actively challenges automated traffic. A residential proxy improves reliability but does not guarantee access. The Actor detects HTTP errors, sign-in/splash/challenge/captcha/error content, and missing cards. It performs bounded fresh-session retries and records unresolved inputs in OUTPUT.failures. If every input fails, the run exits with an error after writing OUTPUT.

Local observations during development: a cold /sch request returned HTTP 403; warming the homepage before search produced current li.s-card active cards, and a 507-card active search succeeded through a residential proxy. These are observations, not uptime guarantees.

Local development

python -m pip install -r requirements.txt
python -m camoufox fetch
pytest -q
python -m compileall -q src tests

Run through Apify CLI/storage emulation with an appropriate INPUT.json, or build the included Dockerfile. The Actor does not require or use eBay credentials.

Responsible use

Scrape only public data, at a reasonable volume, and comply with applicable laws, eBay's terms, and Apify's policies. Avoid collecting personal data you do not need. This project is not affiliated with or endorsed by eBay; names identify compatible marketplaces only.