Lazada Product Scraper - SG, MY, PH, TH, VN, ID

Lazada product search and inventory discovery for SG, MY, PH, TH, VN, ID
with optional product details, buyer reviews, and structured output.

Overview

Scraping runs on the configured worker. Existing input fields and Store pricing are unchanged.

Worker URL is config, not code: the Actor reads WORKER_BASE_URL from Actor environment variables (plus optional per-run workerBaseUrl). Migrating the worker host means updating WORKER_BASE_URL (and WORKER_AUTH if the API key changes) in Apify Console — no Actor code change required. Set WORKER_PROVIDES_PROXY=1 when the worker supplies its own PROXY_URL instead of Apify RESIDENTIAL.

Lazada Product Scraper - SG, MY, PH, TH, VN, ID is a multi-country Apify Actor for collecting Lazada search result listings from:

• Singapore (SG)
• Malaysia (MY)
• Philippines (PH)
• Thailand (TH)
• Vietnam (VN)
• Indonesia (ID)

Search produces keyword-matched listings. Optional detail enrichment supplements the first results with verified product-page data. Inventory discovery is a separate mode: its products are not keyword matches and carry keyword=__sitemap__.

This design is especially useful when you care more about correctness, observability, and market-specific compatibility than maximum raw throughput.

Why use this Actor

• Market research: Track search rankings, titles, prices, ratings, and seller signals across six Lazada markets.
• Competitive monitoring: Compare the same keyword across SG / MY / PH / TH / VN / ID using one consistent output contract.
• Data pipelines: Send normalized records directly into analytics workflows, dashboards, or downstream ETL jobs.
• QA / debugging: Persist RUN_SUMMARY and optional debug artifacts for easier troubleshooting.
• Geo-sensitive collection: The worker requests market-matched egress from the control plane.

What makes this actor different

1) Structured product records

Results include prices, seller information, ratings, and source-provided brand, stock, SKU, category and promotion fields. Missing values remain missing, rather than being inferred.

2) Market-matched egress

The worker resolves the egress country from the selected market:

• SG -> SG
• MY -> MY
• PH -> PH
• TH -> TH
• VN -> VN
• ID -> ID

Egress products and credentials are managed outside the Actor by the control plane.

3) Bounded detail enrichment

enrichMaxItems limits product-detail requests. Details are best-effort: failed enrichment leaves search results intact and is counted in RUN_SUMMARY.diagnostics. Sitemap mode emits only verified products, not bare URLs.

4) Debug-friendly output

Each run can produce:

• dataset records
• RUN_SUMMARY
• INPUT_ECHO
• optional screenshots / HTML / payload metadata in debug mode

Supported markets

Market	Country	Host
SG	Singapore	www.lazada.sg
MY	Malaysia	www.lazada.com.my
PH	Philippines	www.lazada.com.ph
TH	Thailand	www.lazada.co.th
VN	Vietnam	www.lazada.vn
ID	Indonesia	www.lazada.co.id

Input

Main input fields

Field	Type	Description	Default
keywords	string[]	Search keywords. Empty or whitespace-only values are removed before validation.	Required
market	string	Target Lazada market: PH, SG, MY, TH, VN, ID. Store Test uses the lightweight production-proven MY canary.	MY
pages	integer	Search result pages per keyword.	1
maxItems	integer	Most rows for each keyword or explicit seller key, after dedupe (1–200). Default 70; larger requests are collected in parts of 70. Products, reviews, and all other row types share the input budget; the run stops at your maximum total charge. Free Apify plan: 200 rows/run and 10 total runs.	70
minItemsPerPage	integer	Minimum accepted items per parsed AJAX payload.	1
headless	boolean	Run browser headless.	true
includeRawPayload	boolean	Save raw AJAX payloads to KVS.	false
debug	boolean	Save extra debug evidence on failures.	false
maxRetriesPerKeyword	integer	Recoverable retry attempts per keyword.	1
requestTimeoutSecs	integer	Timeout budget per keyword.	90
scrollSteps	integer	Light scroll steps before AJAX fetch.	3
scrollDelayMs	integer	Delay between scroll steps.	300
postLoadWaitMs	integer	Extra wait after page load.	1200
workerBaseUrl	string	Optional per-run override of the Cloud Run worker origin. Default: Actor env WORKER_BASE_URL.	(env)
searchEngine	string	auto, mtop, or browser. Applies to keyword search.	auto
enrichDetails	boolean	Supplement initial results with verified product details.	true
enrichMaxItems	integer	Detail-request limit (0–10).	3
includeReviews	boolean	Emit available buyer reviews as separate rowType=review rows. Product and review rows share maxItems.	false
maxReviewsPerItem	integer	Upper bound on separate review rows considered per product (0–50), still bounded by that input's maxItems.	5
includeSellerInfo	boolean	Attach the seller dimension (shop name, seller rating, follower count, chat availability) to each result via the anonymous store.visit endpoint — no extra credential needed.	false
includeMultiBuy	boolean	Attach the detail.multiBuyItems multi-buy/bundle promo widget to each row when the item participates (totalSize>0). Conditional surface — most catalog items carry no such offer.	false
enrichEngine	string	PDP enrichment channel: auto (free ISP-direct HTML first, then zyte), htmlDirect, zyte, or none.	auto
discoveryMode	string	search, sitemap, both, sellers, live, or shopcatalog. sellers enumerates storefronts from the sitemap-sellers bucket and profiles each via anonymous store.visit. live reads the anonymous live.channel.* feed (live rooms + recommended presenters). shopcatalog walks each seller's shop.category.bysellerkey.query storefront + in-shop category tree (supply sellerKeys to pick specific shops, else sitemap-sellers).	search
sellerKeys	array	Explicit sellerKey list (storefront slug, e.g. itel) for shopcatalog mode. Empty/absent harvests from the sitemap-sellers bucket.	null
sitemapBuckets	integer	Maximum inventory buckets inspected (1–8). In sellers mode this also lifts the leaf cap for full-market enumeration.	3
sitemapMaxUrls	integer	Maximum inventory URLs considered (1–1000), also bounded by item/detail limits.	200

Example input

{
  "keywords": ["phone case", "wireless earbuds"],
  "market": "TH",
  "pages": 1,
  "maxItems": 20,
  "minItemsPerPage": 1,
  "headless": true,
  "includeRawPayload": false,
  "debug": false,
  "maxRetriesPerKeyword": 1,
  "requestTimeoutSecs": 90,
  "scrollSteps": 3,
  "scrollDelayMs": 300,
  "postLoadWaitMs": 1200
}

Output

The actor writes normalized records to the Apify dataset.

Every dataset row is one billed entity. rowType=product rows keep product rating and review-count fields but no embedded review list. With includeReviews=true, each buyer review is a separate rowType=review row with a stable id; the product's itemId, title, productUrl, market, and location fields are copied onto it. All row types share each input's maxItems budget.

Core fields

• keyword
• market
• marketName
• country
• page
• rank
• absoluteRank
• itemId
• skuId
• title
• productUrl
• sourceUrl
• image
• price
• currency
• currencyCode
• capturedAt
• source

Extended fields

Source-provided brandName, brandId, inStock, sku, cheapestSku, skus, categories and promotionId are preserved. Enriched product records also include detailParsed (SKU, product ID, price, currency, availability and available seller/location details). Review rows expose reviewId, buyerId, buyerName, rating, text, boughtDate, and up to 50 images. Requested seller info appears in sellerInfo (shopName, shopRating, followersNumber, isChatShown, storeTabs) plus shopId/sellerKey. Requested multi-buy promos appear in multiBuy (totalSize, type, jumpURL, hasOffers) — only present when the item participates.

Non-product discovery modes emit entity rows instead of product rows. pageType marks the lane: seller (storefront profile: sellerKey, shopId, sellerInfo, shopUrl, storeTabs), live/live_presenter (liveId, liveUuid, roomStatus, anchorId, anchorName, anchorPageUrl, coverImg, jumpUrl, presenterId, presenterName, fansNum, liveRoomCount, actionUrl), store/shop_category (sellerKey, sellerId, shopId, shopName, themeColor, catId, catName, childCount, pcHeaderBGImage, wirelessHeaderBGImage).

• sellerName
• originalPrice
• discount
• ratingScore
• reviewCount
• soldText
• location
• marketHost
• searchUrl
• ajaxUrl
• responseUrl
• responseStatus
• pageType

Example output

{
  "keyword": "phone case",
  "market": "ID",
  "marketName": "Indonesia",
  "country": "ID",
  "page": 1,
  "rank": 1,
  "absoluteRank": 1,
  "itemId": "8777376072",
  "skuId": "16222892938",
  "title": "2 in 1 Case for Oppo...",
  "productUrl": "https://www.lazada.co.id/products/pdp-i8777376072.html",
  "sourceUrl": "https://www.lazada.co.id/products/pdp-i8777376072.html",
  "image": "https://sg-test-11.slatic.net/p/16335a0fd8a0f1051641fd39af5a1014.jpg",
  "price": "Rp17.400",
  "currency": "Rp",
  "currencyCode": "IDR",
  "capturedAt": "2026-06-26T10:56:03.304392+00:00",
  "source": "lazada_browser_ajax",
  "sellerName": "PengKe",
  "ratingScore": 4.4,
  "reviewCount": 10,
  "soldText": "35 sold",
  "location": "Kab. Tangerang",
  "marketHost": "www.lazada.co.id",
  "searchUrl": "https://www.lazada.co.id/catalog/?q=phone+case",
  "ajaxUrl": "https://www.lazada.co.id/tag/phone-case/?q=phone+case&catalog_redirect_tag=true&ajax=true&page=1&isFirstRequest=true",
  "responseUrl": "https://www.lazada.co.id/tag/phone-case/?q=phone+case&catalog_redirect_tag=true&ajax=true&page=1&isFirstRequest=true",
  "responseStatus": 200,
  "pageType": "search"
}

Lead / contact fields

Every record carries the full lead/clue field set defined by the worker contract. On Lazada the verified leads are the seller org and the province/city/district location; everything else is explicitly absent (not gate-locked), and leadAbsences records the reason per subfield.

Field	Type	Meaning on Lazada
phones	[{value, type}]	Per-seller phone: none on Lazada (site-wide revealControls=[], no show-phone control exists). For PH items the array carries the platform hotline (02) 895 29232 with type="platform_hotline" — platform-owned, not a seller contact. Non-PH markets: [].
emails	[{value, type}]	Always [] — no mailto:, plain or obfuscated email anywhere on the platform (helpcenter offers CLEO chat + forms only).
address	{street, unit, locality, region, postalCode, country, full}	Province/city/district level only. street and unit (street number/door plate) are always null — Lazada never exposes street-level addresses. region/locality are filled from the listing location (e.g. "Cavite", TH: "กรุงเทพมหานคร"); PDP enrichment upgrades ship-from granularity to city+district.
geo	{lat, lng}	Always {lat: null, lng: null} — Lazada has no map surface for listings.
person	{name, role}	Always null — no individual names anywhere (sellers are shops, not persons).
org	{name, id}	The verified lead: seller name + seller id from the AJAX mods.listItems[] / PDP seller block (e.g. "Toda Boutique" / 501668704370, TH: "Leader King" / 100191573223).
web	{website, socials[]}	Always empty per-entity ({website: null, socials: []}) — no per-seller website/social links on shop/PDP surfaces. Platform official social links exist but are platform-level, not per-seller.
leadSource	enum	search (list card only) or detail (PDP-enriched record).
leadReveal	enum	visible — all collected leads are already-visible plain data; no reveal interaction exists on Lazada.
leadAbsences	object	Reason per empty subfield (本站无 not-present / 未验证 unverified), matching the 2026-08-29 recon report.

Reveal cost

• No click-reveal anywhere: Lazada exposes no "show phone/email/address" controls on any probed surface (SRP, PDP, shop, helpcenter). There is no reveal XHR to trigger and no reveal-click cost.
• PDP enrichment is the only lead cost: with enrichDetails=true (default) the worker opens the product detail page for up to enrichMaxItems (default 3) results to upgrade org.id, and ship-from city/district granularity. This adds a few seconds per enriched item; a failed enrich degrades gracefully to search-level leads instead of failing the run.
• Proxy and rate-limit impact: the browser-context AJAX + residential proxy path applies as for normal collection; enrichment adds no special proxy requirements.

Market verification status (honest limitations)

Market	Status
PH	Verified in production (2026-08-28 cloud run, build 0.3.7): search + PDP enrichment + platform hotline.
TH	Verified in production (cloud run AkyPcNOOQN1DDD7dJ, Playwright + residential proxy + warmup): full lead set collected, org sample "Leader King"/Bangkok.
MY, SG, VN, ID	Not yet verified in production. During guest-Chrome recon these markets redirected to /tag/ and hit the Alibaba punish iframe — that gate is a guest-environment phenomenon and does not apply to the production path (browser + market-matched residential proxy + warmup, proven working on TH), but no production run has yet confirmed these markets. Collection may work; treat as unverified until a live run succeeds.

Other honest limitations (all also encoded per-item in leadAbsences):

• Per-seller phone, email, street-level address, person name, and social links are absent on this site (site has no reveal controls) — no extraction approach can produce them.
• Platform hotline and platform social links are Lazada-owned platform-level data, not seller contact info.
• This actor is search-result-oriented; it is not a full PDP/shop crawler, and geo coordinates are not available on Lazada.

How it works

1. Validate input and resolve the target market.
2. Open the Lazada storefront in a Playwright browser context.
3. Apply lightweight warmup steps when the selected market needs it.
4. Open the search page for the keyword.
5. Fetch the structured AJAX payload from the in-page context.
6. Parse, normalize, deduplicate, and write dataset records.
7. Save RUN_SUMMARY and related KVS artifacts.

Storage and debug artifacts

Key-value store

Typical keys include:

• INPUT
• INPUT_ECHO
• RUN_SUMMARY

Extra debug keys when debug=true

• ERROR_SUMMARY
• DEBUG_KEYWORD_<slug>_PAGE_<n>_SCREENSHOT
• DEBUG_KEYWORD_<slug>_PAGE_<n>_HTML
• DEBUG_KEYWORD_<slug>_PAGE_<n>_PAYLOAD
• DEBUG_KEYWORD_<slug>_PAGE_<n>_META

Free Apify plan limits (set by this Actor's developer)

Limit	Free Apify plan	Paid Apify plan
Runs of this Actor	10 runs total	Unlimited
Rows per run	Up to 200	Stops at your maximum total charge

When the run cap is hit, the run finishes with a clear status message — not an Apify platform error.

Free Apify plan: up to 10 runs of this Actor and 200 rows per run. Upgrade to a paid Apify plan for unlimited runs. (Limits set by the Actor developer.)

Best practices

• Start with one keyword, pages=1, and low maxItems.
• Market-matched egress is selected by the control plane.
• Keep debug=false for routine runs and enable it only when investigating failures.
• Compare the same keyword across multiple markets to build cross-country price and ranking snapshots.

Local development

Install

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
playwright install chromium

Run tests

$PYTHONPATH=. .venv/bin/pytest -q

Run locally

$PYTHONPATH=. .venv/bin/python -m src.main

Known limitations

• This actor focuses on search result listing collection, not full product detail crawling.
• Long-term stability still depends on market conditions, proxy quality, and Lazada anti-bot behavior.
• Browser-based extraction is more expensive than pure HTTP scraping.

Troubleshooting

Run succeeded but dataset is empty

Check:

• whether the keyword is too narrow
• whether minItemsPerPage is too high
• whether the site returned a challenge or unexpected payload

Market works locally but fails remotely

Check:

• whether the selected market is correct
• worker and control-plane egress diagnostics
• RUN_SUMMARY and debug artifacts in KVS

TH / ID are less stable than other markets

These markets can be stricter. The actor already includes market-specific warmup and punish recovery, but stability can still vary with traffic conditions and IP quality.

I'm on the free Apify plan — are there limits?

Yes. Free Apify plan users get 10 runs of this Actor and up to 200 results per run. These limits are set by the Actor developer (not Apify). When you hit the run cap, the run finishes with a clear status message — it is not a platform bug. Upgrade to a paid Apify plan for unlimited runs.

Reference style used for this README

This README was rewritten in the style of a stronger Apify Store listing: clearer overview, use-case framing, input/output examples, and a visual top section inspired by the presentation style seen on fatihtahta/lazada-scraper, while keeping the feature claims aligned with the actual code in this repository.