DHgate Product Search Scraper (Apify Actor)

Searches DHgate and returns products as structured rows: title, price range, MOQ, supplier, feedback %, URL, image. First learning Actor — DHgate serves everything in a single Next.js __NEXT_DATA__ JSON blob, so it's fetched with plain HTTP (CheerioCrawler, no browser) and parsed from that one object.

How it works

• src/parse.js — pure, unit-tested parsing (productsFromHtml, toRow). Reads props.pageProps.data.totalProducts from the page's __NEXT_DATA__ blob.
• src/main.js — the Actor: builds search.do?searchkey=…&pageNum=N URLs, runs a CheerioCrawler through Apify Proxy, and pushDatas each product.
• test-parse.js — runs the parser against a saved page offline (no account needed).

One-time setup

1. Create a free account at apify.com (free tier ≈ $5/mo credits).
2. Get your token: Console → Settings → Integrations → Personal API token.
3. Install the CLI and log in:

   npm install -g apify-cli
   apify login          # paste the token
   

Proxies — Apify provides them, you configure nothing self-hosted

The proxyConfiguration input uses Apify Proxy (built in). Default is RESIDENTIAL. Locally, apify run routes through Apify Proxy using your login — so even local testing uses real proxies with no Webshare/self-hosted infra.

• Residential (default): safest for e-commerce. Paid per GB, but HTML pages are tiny.
• To try cheaper datacenter proxies, set apifyProxyGroups: [] (or remove the group) in the input. If DHgate starts returning block pages, switch back to RESIDENTIAL.

Test / validate

Offline parser test (fastest, no account):

npm install
node test-parse.js       # expects 50/50 clean rows from sample.html

Local Actor run (uses Apify Proxy via your login):

apify run                # reads storage/key_value_stores/default/INPUT.json
# results land in storage/datasets/default/*.json — inspect them

Default input is { "searchKeywords": ["wood comb"], "maxItemsPerKeyword": 50 }. Edit storage/key_value_stores/default/INPUT.json to change it.

Deploy + cloud test:

$apify push               # builds the image in Apify's cloud

Then in the Console: open the Actor → the input form (from input_schema.json) renders → Start. Check the Dataset tab (table / JSON / CSV) and Log tab. Watch the run status — a low FAILED rate is the buyer-facing health metric.

Publish / monetize (only once it runs reliably)

• Console → Publication → make it Public (appears in the Store).
• Set pay-per-result pricing; you keep 80% of revenue.
• Start unlisted/free to validate, then publish + price.

Output row shape

{ "keyword": "wood comb", "title": "...", "priceMin": 5.07, "priceMax": 16.82,
  "priceRaw": "US $5.07 - 16.82", "moq": 1, "unit": "Piece",
  "url": "https://www.dhgate.com/product/...", "itemCode": "1109697735",
  "seller": "famousbrandbeauty", "sellerUrl": "...", "sellerFeedback": "100.0%",
  "recentlySold": 1, "reviewCount": null, "image": "https://...", "isAd": true }