Lazada Product Scraper - SG, MY, PH, TH, VN, ID

Browser-context Lazada search scraper for SG, MY, PH, TH, VN, ID
with market-matched residential proxies, AJAX extraction, and debug-friendly output.

Overview

Architecture: this Apify Actor is a thin client. Scraping runs on the lazada-com Cloud Run worker (Playwright Chromium + in-page catalog AJAX) over HTTPS + API key. Input/output fields and Store pricing are unchanged.

Worker URL is config, not code: the Actor reads WORKER_BASE_URL from Actor environment variables (plus optional per-run workerBaseUrl). Migrating the worker host means updating WORKER_BASE_URL (and WORKER_AUTH if the API key changes) in Apify Console — no Actor code change required. Set WORKER_PROVIDES_PROXY=1 when the worker supplies its own PROXY_URL instead of Apify RESIDENTIAL.

Lazada Product Scraper - SG, MY, PH, TH, VN, ID is a multi-country Apify Actor for collecting Lazada search result listings from:

• Singapore (SG)
• Malaysia (MY)
• Philippines (PH)
• Thailand (TH)
• Vietnam (VN)
• Indonesia (ID)

The actor does not rely on brittle first-response HTML parsing as its primary data source. Instead, it opens the real Lazada search experience in a browser context, performs lightweight page warmup when needed, calls the site's structured AJAX endpoint, parses the JSON payload, and writes normalized records to the Apify dataset.

This design is especially useful when you care more about correctness, observability, and market-specific compatibility than maximum raw throughput.

Why use this Actor

• Market research: Track search rankings, titles, prices, ratings, and seller signals across six Lazada markets.
• Competitive monitoring: Compare the same keyword across SG / MY / PH / TH / VN / ID using one consistent output contract.
• Data pipelines: Send normalized records directly into analytics workflows, dashboards, or downstream ETL jobs.
• QA / debugging: Persist RUN_SUMMARY and optional debug artifacts for easier troubleshooting.
• Geo-sensitive collection: When Apify Proxy is used, the actor automatically aligns the residential proxy country with the selected market.

What makes this actor different

1) Browser-context AJAX extraction

The actor loads Lazada in a real Playwright browser session and fetches the structured search payload from the in-page context. This is more robust than scraping unstable surface HTML only.

2) Market-matched residential proxies

For Apify runs, the actor automatically resolves the proxy country from the selected market:

• SG -> SG
• MY -> MY
• PH -> PH
• TH -> TH
• VN -> VN
• ID -> ID

When Apify Proxy is used, the actor also enforces RESIDENTIAL proxy routing for the supported remote anti-bot path.

3) Stricter TH / ID handling

For Thailand and Indonesia, the actor includes a site-root warmup path before search and a punish recovery path when needed. This improves stability on stricter storefronts without changing the output schema.

4) Debug-friendly output

Each run can produce:

• dataset records
• RUN_SUMMARY
• INPUT_ECHO
• optional screenshots / HTML / payload metadata in debug mode

Supported markets

Market	Country	Host
SG	Singapore	www.lazada.sg
MY	Malaysia	www.lazada.com.my
PH	Philippines	www.lazada.com.ph
TH	Thailand	www.lazada.co.th
VN	Vietnam	www.lazada.vn
ID	Indonesia	www.lazada.co.id

Input

Main input fields

Field	Type	Description	Default
keywords	string[]	Search keywords. Empty or whitespace-only values are removed before validation.	Required
market	string	Target Lazada market: PH, SG, MY, TH, VN, ID.	PH
pages	integer	Search result pages per keyword.	1
maxItems	integer	Maximum normalized rows written after dedupe (1–200). Free Apify plan: also capped at 200/run and 10 total runs of this Actor.	40
minItemsPerPage	integer	Minimum accepted items per parsed AJAX payload.	1
headless	boolean	Run browser headless.	true
includeRawPayload	boolean	Save raw AJAX payloads to KVS.	false
debug	boolean	Save extra debug evidence on failures.	false
proxyConfiguration	object	Apify Proxy or custom proxy config.	null
maxRetriesPerKeyword	integer	Recoverable retry attempts per keyword.	1
requestTimeoutSecs	integer	Timeout budget per keyword.	90
scrollSteps	integer	Light scroll steps before AJAX fetch.	3
scrollDelayMs	integer	Delay between scroll steps.	300
postLoadWaitMs	integer	Extra wait after page load.	1200
workerBaseUrl	string	Optional per-run override of the Cloud Run worker origin. Default: Actor env WORKER_BASE_URL.	(env)

Example input

{
  "keywords": ["phone case", "wireless earbuds"],
  "market": "TH",
  "pages": 1,
  "maxItems": 20,
  "minItemsPerPage": 1,
  "headless": true,
  "includeRawPayload": false,
  "debug": false,
  "maxRetriesPerKeyword": 1,
  "requestTimeoutSecs": 90,
  "scrollSteps": 3,
  "scrollDelayMs": 300,
  "postLoadWaitMs": 1200,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}

Output

The actor writes normalized records to the Apify dataset.

Core fields

• keyword
• market
• marketName
• country
• page
• rank
• absoluteRank
• itemId
• skuId
• title
• productUrl
• sourceUrl
• image
• price
• currency
• currencyCode
• capturedAt
• source

Extended fields

• sellerName
• originalPrice
• discount
• ratingScore
• reviewCount
• soldText
• location
• marketHost
• searchUrl
• ajaxUrl
• responseUrl
• responseStatus
• pageType

Example output

{
  "keyword": "phone case",
  "market": "ID",
  "marketName": "Indonesia",
  "country": "ID",
  "page": 1,
  "rank": 1,
  "absoluteRank": 1,
  "itemId": "8777376072",
  "skuId": "16222892938",
  "title": "2 in 1 Case for Oppo...",
  "productUrl": "https://www.lazada.co.id/products/pdp-i8777376072.html",
  "sourceUrl": "https://www.lazada.co.id/products/pdp-i8777376072.html",
  "image": "https://sg-test-11.slatic.net/p/16335a0fd8a0f1051641fd39af5a1014.jpg",
  "price": "Rp17.400",
  "currency": "Rp",
  "currencyCode": "IDR",
  "capturedAt": "2026-06-26T10:56:03.304392+00:00",
  "source": "lazada_browser_ajax",
  "sellerName": "PengKe",
  "ratingScore": 4.4,
  "reviewCount": 10,
  "soldText": "35 sold",
  "location": "Kab. Tangerang",
  "marketHost": "www.lazada.co.id",
  "searchUrl": "https://www.lazada.co.id/catalog/?q=phone+case",
  "ajaxUrl": "https://www.lazada.co.id/tag/phone-case/?q=phone+case&catalog_redirect_tag=true&ajax=true&page=1&isFirstRequest=true",
  "responseUrl": "https://www.lazada.co.id/tag/phone-case/?q=phone+case&catalog_redirect_tag=true&ajax=true&page=1&isFirstRequest=true",
  "responseStatus": 200,
  "pageType": "search"
}

How it works

1. Validate input and resolve the target market.
2. Open the Lazada storefront in a Playwright browser context.
3. Apply lightweight warmup steps when the selected market needs it.
4. Open the search page for the keyword.
5. Fetch the structured AJAX payload from the in-page context.
6. Parse, normalize, deduplicate, and write dataset records.
7. Save RUN_SUMMARY and related KVS artifacts.

Storage and debug artifacts

Key-value store

Typical keys include:

• INPUT
• INPUT_ECHO
• RUN_SUMMARY

Extra debug keys when debug=true

• ERROR_SUMMARY
• DEBUG_KEYWORD_<slug>_PAGE_<n>_SCREENSHOT
• DEBUG_KEYWORD_<slug>_PAGE_<n>_HTML
• DEBUG_KEYWORD_<slug>_PAGE_<n>_PAYLOAD
• DEBUG_KEYWORD_<slug>_PAGE_<n>_META

Free Apify plan limits (set by this Actor's developer)

Limit	Free Apify plan	Paid Apify plan
Runs of this Actor	10 runs total	Unlimited
Results per run (maxItems)	Up to 200	Up to 200

When the run cap is hit, the run finishes with a clear status message — not an Apify platform error.

Free Apify plan: up to 10 runs of this Actor and 200 results per run. Upgrade to a paid Apify plan for unlimited runs. (Limits set by the Actor developer.)

Best practices

• Start with one keyword, pages=1, and low maxItems.
• Use Apify residential proxies for remote runs.
• Keep debug=false for routine runs and enable it only when investigating failures.
• Compare the same keyword across multiple markets to build cross-country price and ranking snapshots.

Local development

Install

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
playwright install chromium

Run tests

$PYTHONPATH=. .venv/bin/pytest -q

Run locally

$PYTHONPATH=. .venv/bin/python -m src.main

Known limitations

• This actor focuses on search result listing collection, not full product detail crawling.
• Long-term stability still depends on market conditions, proxy quality, and Lazada anti-bot behavior.
• Browser-based extraction is more expensive than pure HTTP scraping.

Troubleshooting

Run succeeded but dataset is empty

Check:

• whether the keyword is too narrow
• whether minItemsPerPage is too high
• whether the site returned a challenge or unexpected payload

Market works locally but fails remotely

Check:

• proxy settings
• whether Apify Proxy is enabled
• whether the selected market matches the proxy country
• RUN_SUMMARY and debug artifacts in KVS

TH / ID are less stable than other markets

These markets can be stricter. The actor already includes market-specific warmup and punish recovery, but stability can still vary with traffic conditions and IP quality.

I'm on the free Apify plan — are there limits?

Yes. Free Apify plan users get 10 runs of this Actor and up to 200 results per run. These limits are set by the Actor developer (not Apify). When you hit the run cap, the run finishes with a clear status message — it is not a platform bug. Upgrade to a paid Apify plan for unlimited runs.

Reference style used for this README

This README was rewritten in the style of a stronger Apify Store listing: clearer overview, use-case framing, input/output examples, and a visual top section inspired by the presentation style seen on fatihtahta/lazada-scraper, while keeping the feature claims aligned with the actual code in this repository.