Lazada Product Scraper - SG, MY, PH, TH, VN, ID

Browser-context Lazada search scraper for SG, MY, PH, TH, VN, ID
with market-matched residential proxies, AJAX extraction, and debug-friendly output.

Overview

Architecture: this Apify Actor is a thin client. Scraping runs on the lazada-com Cloud Run worker (Playwright Chromium + in-page catalog AJAX) over HTTPS + API key. Input/output fields and Store pricing are unchanged.

Worker URL is config, not code: the Actor reads WORKER_BASE_URL from Actor environment variables (plus optional per-run workerBaseUrl). Migrating the worker host means updating WORKER_BASE_URL (and WORKER_AUTH if the API key changes) in Apify Console — no Actor code change required. Set WORKER_PROVIDES_PROXY=1 when the worker supplies its own PROXY_URL instead of Apify RESIDENTIAL.

Lazada Product Scraper - SG, MY, PH, TH, VN, ID is a multi-country Apify Actor for collecting Lazada search result listings from:

• Singapore (SG)
• Malaysia (MY)
• Philippines (PH)
• Thailand (TH)
• Vietnam (VN)
• Indonesia (ID)

The actor does not rely on brittle first-response HTML parsing as its primary data source. Instead, it opens the real Lazada search experience in a browser context, performs lightweight page warmup when needed, calls the site's structured AJAX endpoint, parses the JSON payload, and writes normalized records to the Apify dataset.

This design is especially useful when you care more about correctness, observability, and market-specific compatibility than maximum raw throughput.

Why use this Actor

• Market research: Track search rankings, titles, prices, ratings, and seller signals across six Lazada markets.
• Competitive monitoring: Compare the same keyword across SG / MY / PH / TH / VN / ID using one consistent output contract.
• Data pipelines: Send normalized records directly into analytics workflows, dashboards, or downstream ETL jobs.
• QA / debugging: Persist RUN_SUMMARY and optional debug artifacts for easier troubleshooting.
• Geo-sensitive collection: When Apify Proxy is used, the actor automatically aligns the residential proxy country with the selected market.

What makes this actor different

1) Browser-context AJAX extraction

The actor loads Lazada in a real Playwright browser session and fetches the structured search payload from the in-page context. This is more robust than scraping unstable surface HTML only.

2) Market-matched residential proxies

For Apify runs, the actor automatically resolves the proxy country from the selected market:

• SG -> SG
• MY -> MY
• PH -> PH
• TH -> TH
• VN -> VN
• ID -> ID

When Apify Proxy is used, the actor also enforces RESIDENTIAL proxy routing for the supported remote anti-bot path.

3) Stricter TH / ID handling

For Thailand and Indonesia, the actor includes a site-root warmup path before search and a punish recovery path when needed. This improves stability on stricter storefronts without changing the output schema.

4) Debug-friendly output

Each run can produce:

• dataset records
• RUN_SUMMARY
• INPUT_ECHO
• optional screenshots / HTML / payload metadata in debug mode

Supported markets

Market	Country	Host
SG	Singapore	www.lazada.sg
MY	Malaysia	www.lazada.com.my
PH	Philippines	www.lazada.com.ph
TH	Thailand	www.lazada.co.th
VN	Vietnam	www.lazada.vn
ID	Indonesia	www.lazada.co.id

Input

Main input fields

Field	Type	Description	Default
keywords	string[]	Search keywords. Empty or whitespace-only values are removed before validation.	Required
market	string	Target Lazada market: PH, SG, MY, TH, VN, ID. Store Test uses TH (PH catalog AJAX is currently punish-prone).	TH
pages	integer	Search result pages per keyword.	1
maxItems	integer	Maximum normalized rows written after dedupe (1–200). Free Apify plan: also capped at 200/run and 10 total runs of this Actor. Store Test uses 1.	1
minItemsPerPage	integer	Minimum accepted items per parsed AJAX payload.	1
headless	boolean	Run browser headless.	true
includeRawPayload	boolean	Save raw AJAX payloads to KVS.	false
debug	boolean	Save extra debug evidence on failures.	false
proxyConfiguration	object	Apify Proxy or custom proxy config.	null
maxRetriesPerKeyword	integer	Recoverable retry attempts per keyword.	1
requestTimeoutSecs	integer	Timeout budget per keyword.	90
scrollSteps	integer	Light scroll steps before AJAX fetch.	3
scrollDelayMs	integer	Delay between scroll steps.	300
postLoadWaitMs	integer	Extra wait after page load.	1200
workerBaseUrl	string	Optional per-run override of the Cloud Run worker origin. Default: Actor env WORKER_BASE_URL.	(env)

Example input

{
  "keywords": ["phone case", "wireless earbuds"],
  "market": "TH",
  "pages": 1,
  "maxItems": 20,
  "minItemsPerPage": 1,
  "headless": true,
  "includeRawPayload": false,
  "debug": false,
  "maxRetriesPerKeyword": 1,
  "requestTimeoutSecs": 90,
  "scrollSteps": 3,
  "scrollDelayMs": 300,
  "postLoadWaitMs": 1200,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}

Output

The actor writes normalized records to the Apify dataset.

Core fields

• keyword
• market
• marketName
• country
• page
• rank
• absoluteRank
• itemId
• skuId
• title
• productUrl
• sourceUrl
• image
• price
• currency
• currencyCode
• capturedAt
• source

Extended fields

• sellerName
• originalPrice
• discount
• ratingScore
• reviewCount
• soldText
• location
• marketHost
• searchUrl
• ajaxUrl
• responseUrl
• responseStatus
• pageType

Example output

{
  "keyword": "phone case",
  "market": "ID",
  "marketName": "Indonesia",
  "country": "ID",
  "page": 1,
  "rank": 1,
  "absoluteRank": 1,
  "itemId": "8777376072",
  "skuId": "16222892938",
  "title": "2 in 1 Case for Oppo...",
  "productUrl": "https://www.lazada.co.id/products/pdp-i8777376072.html",
  "sourceUrl": "https://www.lazada.co.id/products/pdp-i8777376072.html",
  "image": "https://sg-test-11.slatic.net/p/16335a0fd8a0f1051641fd39af5a1014.jpg",
  "price": "Rp17.400",
  "currency": "Rp",
  "currencyCode": "IDR",
  "capturedAt": "2026-06-26T10:56:03.304392+00:00",
  "source": "lazada_browser_ajax",
  "sellerName": "PengKe",
  "ratingScore": 4.4,
  "reviewCount": 10,
  "soldText": "35 sold",
  "location": "Kab. Tangerang",
  "marketHost": "www.lazada.co.id",
  "searchUrl": "https://www.lazada.co.id/catalog/?q=phone+case",
  "ajaxUrl": "https://www.lazada.co.id/tag/phone-case/?q=phone+case&catalog_redirect_tag=true&ajax=true&page=1&isFirstRequest=true",
  "responseUrl": "https://www.lazada.co.id/tag/phone-case/?q=phone+case&catalog_redirect_tag=true&ajax=true&page=1&isFirstRequest=true",
  "responseStatus": 200,
  "pageType": "search"
}

Lead / contact fields

Every record carries the full lead/clue field set defined by the worker contract. On Lazada the verified leads are the seller org and the province/city/district location; everything else is explicitly absent (not gate-locked), and leadAbsences records the reason per subfield.

Field	Type	Meaning on Lazada
phones	[{value, type}]	Per-seller phone: none on Lazada (site-wide revealControls=[], no show-phone control exists). For PH items the array carries the platform hotline (02) 895 29232 with type="platform_hotline" — platform-owned, not a seller contact. Non-PH markets: [].
emails	[{value, type}]	Always [] — no mailto:, plain or obfuscated email anywhere on the platform (helpcenter offers CLEO chat + forms only).
address	{street, unit, locality, region, postalCode, country, full}	Province/city/district level only. street and unit (street number/door plate) are always null — Lazada never exposes street-level addresses. region/locality are filled from the listing location (e.g. "Cavite", TH: "กรุงเทพมหานคร"); PDP enrichment upgrades ship-from granularity to city+district.
geo	{lat, lng}	Always {lat: null, lng: null} — Lazada has no map surface for listings.
person	{name, role}	Always null — no individual names anywhere (sellers are shops, not persons).
org	{name, id}	The verified lead: seller name + seller id from the AJAX mods.listItems[] / PDP seller block (e.g. "Toda Boutique" / 501668704370, TH: "Leader King" / 100191573223).
web	{website, socials[]}	Always empty per-entity ({website: null, socials: []}) — no per-seller website/social links on shop/PDP surfaces. Platform official social links exist but are platform-level, not per-seller.
leadSource	enum	search (list card only) or detail (PDP-enriched record).
leadReveal	enum	visible — all collected leads are already-visible plain data; no reveal interaction exists on Lazada.
leadAbsences	object	Reason per empty subfield (本站无 not-present / 未验证 unverified), matching the 2026-08-29 recon report.

Reveal cost

• No click-reveal anywhere: Lazada exposes no "show phone/email/address" controls on any probed surface (SRP, PDP, shop, helpcenter). There is no reveal XHR to trigger and no reveal-click cost.
• PDP enrichment is the only lead cost: with enrichDetails=true (default) the worker opens the product detail page for up to enrichMaxItems (default 3) results to upgrade org.id, and ship-from city/district granularity. This adds a few seconds per enriched item; a failed enrich degrades gracefully to search-level leads instead of failing the run.
• Proxy and rate-limit impact: the browser-context AJAX + residential proxy path applies as for normal collection; enrichment adds no special proxy requirements.

Market verification status (honest limitations)

Market	Status
PH	Verified in production (2026-08-28 cloud run, build 0.3.7): search + PDP enrichment + platform hotline.
TH	Verified in production (cloud run AkyPcNOOQN1DDD7dJ, Playwright + residential proxy + warmup): full lead set collected, org sample "Leader King"/Bangkok.
MY, SG, VN, ID	Not yet verified in production. During guest-Chrome recon these markets redirected to /tag/ and hit the Alibaba punish iframe — that gate is a guest-environment phenomenon and does not apply to the production path (browser + market-matched residential proxy + warmup, proven working on TH), but no production run has yet confirmed these markets. Collection may work; treat as unverified until a live run succeeds.

Other honest limitations (all also encoded per-item in leadAbsences):

• Per-seller phone, email, street-level address, person name, and social links are absent on this site (site has no reveal controls) — no extraction approach can produce them.
• Platform hotline and platform social links are Lazada-owned platform-level data, not seller contact info.
• This actor is search-result-oriented; it is not a full PDP/shop crawler, and geo coordinates are not available on Lazada.

How it works

1. Validate input and resolve the target market.
2. Open the Lazada storefront in a Playwright browser context.
3. Apply lightweight warmup steps when the selected market needs it.
4. Open the search page for the keyword.
5. Fetch the structured AJAX payload from the in-page context.
6. Parse, normalize, deduplicate, and write dataset records.
7. Save RUN_SUMMARY and related KVS artifacts.

Storage and debug artifacts

Key-value store

Typical keys include:

• INPUT
• INPUT_ECHO
• RUN_SUMMARY

Extra debug keys when debug=true

• ERROR_SUMMARY
• DEBUG_KEYWORD_<slug>_PAGE_<n>_SCREENSHOT
• DEBUG_KEYWORD_<slug>_PAGE_<n>_HTML
• DEBUG_KEYWORD_<slug>_PAGE_<n>_PAYLOAD
• DEBUG_KEYWORD_<slug>_PAGE_<n>_META

Free Apify plan limits (set by this Actor's developer)

Limit	Free Apify plan	Paid Apify plan
Runs of this Actor	10 runs total	Unlimited
Results per run (maxItems)	Up to 200	Up to 200

When the run cap is hit, the run finishes with a clear status message — not an Apify platform error.

Free Apify plan: up to 10 runs of this Actor and 200 results per run. Upgrade to a paid Apify plan for unlimited runs. (Limits set by the Actor developer.)

Best practices

• Start with one keyword, pages=1, and low maxItems.
• Use Apify residential proxies for remote runs.
• Keep debug=false for routine runs and enable it only when investigating failures.
• Compare the same keyword across multiple markets to build cross-country price and ranking snapshots.

Local development

Install

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
playwright install chromium

Run tests

$PYTHONPATH=. .venv/bin/pytest -q

Run locally

$PYTHONPATH=. .venv/bin/python -m src.main

Known limitations

• This actor focuses on search result listing collection, not full product detail crawling.
• Long-term stability still depends on market conditions, proxy quality, and Lazada anti-bot behavior.
• Browser-based extraction is more expensive than pure HTTP scraping.

Troubleshooting

Run succeeded but dataset is empty

Check:

• whether the keyword is too narrow
• whether minItemsPerPage is too high
• whether the site returned a challenge or unexpected payload

Market works locally but fails remotely

Check:

• proxy settings
• whether Apify Proxy is enabled
• whether the selected market matches the proxy country
• RUN_SUMMARY and debug artifacts in KVS

TH / ID are less stable than other markets

These markets can be stricter. The actor already includes market-specific warmup and punish recovery, but stability can still vary with traffic conditions and IP quality.

I'm on the free Apify plan — are there limits?

Yes. Free Apify plan users get 10 runs of this Actor and up to 200 results per run. These limits are set by the Actor developer (not Apify). When you hit the run cap, the run finishes with a clear status message — it is not a platform bug. Upgrade to a paid Apify plan for unlimited runs.

Reference style used for this README

This README was rewritten in the style of a stronger Apify Store listing: clearer overview, use-case framing, input/output examples, and a visual top section inspired by the presentation style seen on fatihtahta/lazada-scraper, while keeping the feature claims aligned with the actual code in this repository.