Alibaba Category Scraper

Alibaba Category Scraper

Browser-first extraction for public Alibaba.com category, search, catalog, and product-detail pages. The Actor collects publicly exposed product and supplier facts from the rendered page, embedded offer data, SSR cards, and Product/ItemList JSON-LD. It supports multiple sources, bounded pagination, deduplication, filters, sorting, standard Apify Proxy routing, and fail-closed diagnostics.

The Actor is not affiliated with Alibaba Group. It requests only public pages, does not sign in, solve CAPTCHAs, evade WAFs, rotate identities to bypass access controls, or call private APIs. When a public response is challenged or unavailable, it stops at that boundary and reports a bounded diagnostic instead of fabricating a product. Availability can vary by region, session, and Cloud transport.

Why use this Actor

Use it to turn public Alibaba category or search pages into analysis-ready product rows for supplier discovery, preliminary sourcing research, price/MOQ comparisons, catalog monitoring, and market reconnaissance. Multiple input URLs can be crawled in one bounded run, and filters can narrow the result set before the global maxItems limit is applied.

What is extracted

When the public response exposes the value, product rows can include:

• identity: url, productUrl, productId, productName, title
• pricing: price, priceCurrency, priceFormatted, priceMin, priceMax, priceUnit, discount, promotionPrice
• order terms: minOrder, minOrderUnit, minOrderFormatted
• supplier: supplierName, supplierProfileUrl, supplierCountry, supplierLocation, supplierYears, supplierRating, supplierRatingValue, supplierId
• engagement: ratingsCount, ratingsCountValue, soldCount, soldCountValue, transactionCount, transactionCountValue
• catalog facts: category, brand, availability, productScore, supplierServiceScore, shippingScore
• media and merchandising: images, thumbnail, imageCount, badges, certifications, sellingPoints, isAd, isCertified
• provenance: sourceUrl, pageNumber, position, scrapedAt

Missing values are omitted rather than fabricated. thumbnail is the first URL in images when media is enabled. Image output is restricted to public Alibaba CDN assets, with obvious logo, icon, avatar, placeholder, and loading assets removed. Operational counts and failure evidence are written to the OUTPUT_SUMMARY key-value record rather than mixed into product rows.

Input

Keyword search

The simplest input creates a public Alibaba search URL:

{
  "keyword": "wireless earbuds",
  "maxItems": 25,
  "maxPages": 3,
  "sortBy": "best_match"
}

Multiple category or search sources

Use explicit public URLs when you already have them. All sources share one global maxItems limit:

{
  "startUrls": [
    "https://www.alibaba.com/trade/search?SearchText=wireless+earbuds&has4Tab=true&tab=all",
    "https://www.alibaba.com/trade/search?SearchText=bluetooth+speakers&has4Tab=true&tab=all"
  ],
  "maxItems": 50,
  "maxPages": 0,
  "deduplicate": true,
  "includeMedia": true,
  "includeDiagnostics": true
}

maxPages: 0 means automatic pagination with a hard 100-page safety cap. Pages within one source are sequential; maxConcurrency controls how many source URLs can run at once.

Filters and sorting

{
  "keyword": "led work light",
  "maxItems": 100,
  "maxPages": 5,
  "sortBy": "price_asc",
  "minPrice": 5,
  "maxPrice": 40,
  "minOrderQuantity": 10,
  "minSupplierRating": 4.5,
  "minReviewCount": 20,
  "minSoldCount": 100,
  "supplierCountry": "CN",
  "certifiedOnly": true,
  "requiredBadges": ["CE"],
  "excludeAds": true
}

Supported sortBy values are best_match, price_asc, price_desc, rating_desc, reviews_desc, sold_desc, and supplier_years_desc.

Developer and proxy options

{
  "keyword": "wireless earbuds",
  "requestDelayMs": 1200,
  "maxConcurrency": 2,
  "maxRequestRetries": 3,
  "requestTimeoutSecs": 60,
  "requestHandlerTimeoutSecs": 300,
  "selectorTimeoutSecs": 30,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}

Proxy configuration changes only the network route. It does not bypass a challenge or access control. includeDiagnostics: false suppresses diagnostic rows in the dataset, but diagnosticsFound and the blocked or partial status remain in OUTPUT_SUMMARY.

Complete input reference

The supported canonical fields are:

• keyword: search phrase; defaults to wireless earbuds when no source URL is supplied.
• startUrls: up to 20 public HTTPS Alibaba URLs.
• categoryUrl and categoryUrls: convenience fields appended after startUrls.
• maxItems: global emitted product-row limit, 1–500; default 40.
• maxPages: pages per source, 0–100; default 3. Zero enables bounded automatic pagination.
• sortBy: best_match, price_asc, price_desc, rating_desc, reviews_desc, sold_desc, or supplier_years_desc.
• includeMedia, deduplicate, and includeDiagnostics: booleans, all defaulting to true.
• excludeAds and certifiedOnly: booleans, default false.
• supplierCountry, supplierLocation, minPrice, maxPrice, minOrderQuantity, minSupplierRating, minReviewCount, minSoldCount, and requiredBadges: optional filters.
• requestDelayMs: 0–15,000 ms; default 1,000.
• maxConcurrency: 1–5 source workers; default 2.
• maxRequestRetries: 0–10; default 3.
• requestTimeoutSecs: 15–300 seconds; default 60.
• requestHandlerTimeoutSecs: 60–900 seconds; default 300.
• selectorTimeoutSecs: 5–120 seconds; default 30.
• proxyConfiguration: optional useApifyProxy, apifyProxyGroups, and apifyProxyCountry.
• fixtureFile: an Actor-relative .html fixture for local QA only; Cloud runs reject it.

For compatibility, the runtime also accepts legacy aliases such as keywords, searchQuery, query, queries, sourceUrl, sourceUrls, maxProducts, max_products, max_pages, pageDelayMs, navigationTimeoutSecs, timeoutSecs, and timeoutMs. Canonical fields are recommended for new runs.

Deterministic local QA

The bundled fixture is for local validation and cannot be used to create Cloud output:

{
  "startUrls": ["https://www.alibaba.com/"],
  "maxItems": 1,
  "maxPages": 1,
  "includeMedia": true,
  "requestDelayMs": 0,
  "maxConcurrency": 1,
  "maxRequestRetries": 0,
  "requestTimeoutSecs": 15,
  "fixtureFile": "fixtures/sample.html"
}

For pagination fixtures, page 2 and later are loaded from sibling files named sample-page-2.html, sample-page-3.html, and so on.

You can download the dataset in various formats such as JSON, HTML, CSV, or Excel.

Run it in Apify Console

1. Open the Actor and select the Input tab.
2. Paste a canonical JSON input, such as the keyword example above, or enter the fields in the form.
3. Select Save & Run (or Start) and wait for the run to finish.
4. Open the Dataset tab to inspect product rows and any emitted diagnostics.
5. Open the Key-value store tab and inspect OUTPUT_SUMMARY for status, counts, pagination, filters, and failure evidence.
6. Use the Dataset export controls to download JSON, HTML, CSV, or Excel.

Product output example

The following rich row mirrors the bundled public-shape fixture. Live values vary and are emitted only when observed in the public response:

{
  "url": "https://www.alibaba.com/product-detail/rechargeable-led-work-light_1234567890123.html",
  "productId": "1234567890123",
  "productName": "Rechargeable LED Work Light",
  "title": "Rechargeable LED Work Light",
  "productUrl": "https://www.alibaba.com/product-detail/rechargeable-led-work-light_1234567890123.html",
  "price": 12.5,
  "priceCurrency": "USD",
  "priceFormatted": "12.50",
  "priceMin": 12.5,
  "priceMax": 12.5,
  "category": "Portable Lighting",
  "brand": "BrightForge",
  "availability": "https://schema.org/InStock",
  "supplierName": "BrightForge Industrial Co.",
  "supplierRatingValue": 4.8,
  "ratingsCountValue": 38,
  "images": ["https://s.alicdn.com/@sc04/kf/LED-work-light.jpg"],
  "thumbnail": "https://s.alicdn.com/@sc04/kf/LED-work-light.jpg",
  "imageCount": 1,
  "sourceUrl": "https://www.alibaba.com/",
  "pageNumber": 1,
  "position": 1,
  "scrapedAt": "2026-09-05T00:00:00.000Z"
}

HTTP fallback output example

If browser extraction fails before producing records, the bounded HTTP fallback uses the same product-row contract. It does not add a fake transport field or claim fields that were not present:

{
  "url": "https://www.alibaba.com/product-detail/example-public-item_123456789.html",
  "productId": "123456789",
  "productName": "Example public item",
  "productUrl": "https://www.alibaba.com/product-detail/example-public-item_123456789.html",
  "price": 4.25,
  "priceCurrency": "USD",
  "priceFormatted": "4.25",
  "sourceUrl": "https://www.alibaba.com/trade/search?SearchText=example",
  "pageNumber": 1,
  "position": 1,
  "scrapedAt": "2026-09-08T00:00:00.000Z"
}

Diagnostic output example

Blocked or empty sources use a minimal four-field diagnostic shape:

{
  "url": "https://www.alibaba.com/trade/search?SearchText=wireless+earbuds",
  "error": "Alibaba presented an access-control or challenge response.",
  "errorCode": "BLOCKED_SOURCE",
  "scrapedAt": "2026-09-08T00:00:00.000Z"
}

Run summary example

OUTPUT_SUMMARY contains operational evidence without changing the dataset row contract:

{
  "status": "SUCCEEDED",
  "keyword": "wireless earbuds",
  "sourceCount": 1,
  "maxItems": 25,
  "maxPages": 3,
  "pagesFetched": 2,
  "productCount": 2,
  "successfulCount": 2,
  "itemCount": 2,
  "diagnosticCount": 0,
  "diagnosticsFound": 0,
  "dataAvailable": true,
  "duplicateCount": 0,
  "filteredOutCount": 0,
  "usedProxy": false,
  "fallbackUsed": false,
  "finishedAt": "2026-09-08T00:00:00.000Z"
}

Cost

The run uses Apify compute time and a Chromium browser. Enabling Apify Proxy can add proxy usage charges according to your Apify plan. Actual cost depends on memory, run duration, page count, retries, and proxy routing; review the run cost shown by Apify for the authoritative amount.

Troubleshooting, API, and Issues

• BLOCKED_SOURCE, HTTP_401, HTTP_403, HTTP_429, HTTP_451, and BROWSER_LAUNCH_FAILED indicate a bounded access or runtime failure; inspect failureEvidence in OUTPUT_SUMMARY.
• NO_PRODUCT_EVIDENCE means the public response was readable but did not expose a supported Product, offer-list, SSR-card, or JSON-LD record.
• Try a smaller page limit, a respectful requestDelayMs, or a different public category URL. Proxy routing may change availability but is not an access-control bypass.
• Use the Actor API tab for the run endpoint and input schema. The Dataset API exposes product rows, while the key-value-store API exposes OUTPUT_SUMMARY.
• Report reproducible Actor problems through the Actor Issues tab with the run URL, sanitized input, and relevant errorCode; do not include credentials or private data.

Privacy, legal, and responsible use

Use only public pages and information you are authorized to collect. Respect Alibaba's terms, robots guidance, applicable privacy and data-protection laws, rate limits, and any supplier or image rights. Do not use this Actor to collect private account data, bypass login or CAPTCHA controls, or create misleading commercial claims. You are responsible for validating supplier information and for how exported data is used.

Run and validate locally

From this Actor directory:

npm install
npm test
apify validate-schema
apify run --purge --input-file .actor/input.json
npm run validate

npm run validate checks that product rows have canonical Alibaba product URLs, no debugging wrapper fields, safe product-scoped images, and the thumbnail === images[0] invariant. The summary distinguishes emitted diagnostics (diagnosticCount) from all observed failures (diagnosticsFound).