Scrape TikTok Shop in 6 markets (US, SG, MY, TH, VN, PH): keyword search, categories, full product details with variants, reviews one-per-row, store analytics with GMV estimates and creator profiles. Local prices and currencies. No login, no API key, no proxy setup. Clean JSON, CSV and Excel export.
Six marketplaces instead of one. A new Region dropdown — the first field on the form — covers 🇺🇸 US, 🇸🇬 Singapore, 🇲🇾 Malaysia, 🇹🇭 Thailand, 🇻🇳 Vietnam and 🇵🇭 Philippines: the storefronts TikTok publishes as hreflang whose keyword search it actually serves on the web. The UK, Japan and Mexico render fine but have no web search (the UK edge routes no search endpoint; Japan and Mexico answer every keyword with an empty result set), so they are left out rather than shipped half-working — along with Brazil (login wall for anonymous visitors), Indonesia (a different site, shop-id.tokopedia.com) and the EU markets (app-only, no web storefront at all). Category listings (with full pagination), product pages and reviews were verified live in all nine; Search and Store stay US-only because those two routes exist only on TikTok's US edge — shop.tiktok.com/gb/s?q=... and /th/store/... answer a bare 404, and so does a US store URL requested from a Singapore IP. Picking an unsupported mode for a region now fails immediately with a message naming the regions that do serve it, instead of scraping nothing for five minutes.
Keyword search reaches 6 of the 9 markets, and goes deeper everywhere. There is no search page outside the US, but the feed endpoint behind it — POST /api/shop/brandy_desktop/s/product_list — is routed on every edge except the EU one, so search no longer needs the page. It works in the US, Singapore, Malaysia, Thailand, Vietnam and the Philippines. The keyword travels in load_more_params.search_word, and one call returns up to 199 products (count=200; anything higher answers empty, and offset/cursor/page_token are echoed back unchanged, so 199 is the whole result set). That also lifts the US ceiling: a US keyword used to yield 20-50 products across ten server-rendered pages, and now returns up to 199 in one request. A sorted US search still takes the old page, which is the only surface that applies a sort order. Three markets have no web search and are marked as such in the input: the UK edge routes neither the search page nor the feed endpoint (404 on both, and on the suggestion endpoint and the H5 view too), and Japan and Mexico accept the request but answer with an empty result set for every keyword and every request shape tried.
The WAF cookies are what blocked search outside the US. A storefront page hands out three cookies: ttwid, plus dkms-token and dkms-type, which carry the edge WAF's verdict on the session. On every non-US edge the search feed refuses any request that presents that pair — code=10000, on every exit IP, from two different proxy providers, signed or not — and serves the very same request with ttwid alone. Measured: full jar 0/6, ttwid only 5/5 with 140-180 products of the market's own catalogue. The search request now drops those two cookies (it has to clear the jar: curl_cffi merges a per-request cookies= mapping into the session's jar rather than replacing it). US is the opposite and keeps its full jar, so the policy is per region.
Search is never signed. Escalating to X-Tts-Oec-Bsid does not merely fail to help on this endpoint, it breaks it: in a paired test on one session the plain request returned 20 products and the signed one that followed answered code=10000. Twice.
Search calls are paced, and a throttle is waited out instead of retried into. TikTok rate-limits the search feed per marketplace, and the limit is not per exit IP — 24 distinct residential IPs were refused together, two unrelated proxy providers behaved identically, and the same endpoint answered normally minutes later at a slower cadence. The Actor's usual answer to a rejection (drop the session, take a new IP, try again immediately, ten times over) is therefore exactly what causes it, so search now issues one call at a time a few seconds apart, waits 20 s and then 45 s in the same session if it is told to back off, and reports the keyword as throttled instead of hammering. Everything else keeps the old session-rotation behaviour, which is right for the risk control it actually faces.
The WAF cookies were what blocked search outside the US. A storefront page hands out three cookies: ttwid, plus dkms-token and dkms-type, which carry the edge WAF's verdict on the session. Every non-US edge refuses a search request that presents that pair — code=10000, on every exit IP, from two different proxy providers, signed or not — and serves the very same request with ttwid alone. Measured: full jar 0/6, ttwid only 5/5 with 140-180 products of the market's own catalogue. The search request now clears those two out of the session jar; it has to be the jar, because curl_cffi merges a per-request cookies= mapping into it rather than replacing it. The US edge is the opposite and keeps its full jar, so the policy is per region.
Search is never signed. Escalating to X-Tts-Oec-Bsid does not merely fail to help on this endpoint, it breaks it: in a paired test on a single session the plain request returned 20 products and the signed one that followed answered code=10000. Twice over.
The request signer now speaks all three of TikTok's security stacks. TikTok runs separate web-security services per edge — ttp for the US, sg for SEA/JP/MX, eu for the UK — each with its own unisec loader, core, bssdk and webmssdk bundle and its own msToken host. The sidecar had the US bundles hardcoded, so every signed request made for a non-US storefront carried a signature the wrong service would never accept. All three stacks are now vendored under signer/sdk/<stack>/ and the sidecar is spawned on the one belonging to the region, with the mocked page pointing at that storefront. This is what makes escalation meaningful outside the US at all.
The proxy field is gone. TikTok blocks datacenter IPs and serves each storefront only to its own country, which left exactly one workable setting — so the Actor now picks it: Apify Residential in the region's country, one configuration per country, a fresh IP per session. Existing runs that still pass proxyConfiguration keep working; the value is simply ignored.
A pasted URL now decides its own market.https://shop.tiktok.com/gb/store/... is scraped from UK IPs even if the dropdown was left on United States, and a run may mix markets — each query gets the right country. A URL naming a market the Actor cannot reach is rejected up front with the list of supported ones.
Creator scrapes always run on the US pool. Creator profiles come from www.tiktok.com, which answers most non-US residential exits with a WAF challenge instead of the profile (measured 6/6 on Japanese, Mexican and SEA exits, 0/5 on US). That one mode is now pinned to US IPs regardless of the selected region, so it keeps working while the rest of the run stays in its market.
Every row says which market it came from. New storefront (us, gb, th, …) and currency fields on every product, review and store record, and as columns in the dataset views — prices arrive as bare numbers, so a dataset mixing markets was previously ambiguous. Creator rows are left alone: they have no market and no prices.
Session warmup is now a per-market setting. The right behaviour genuinely differs: on the UK storefront a cold product page hits "Security Check" on about two thirds of products and loads reliably once the session has visited /gb, while on Thailand the reverse is true (a warmed session captcha'd 8/8, cold passed 3/3). Each region carries the measured answer instead of one global rule.
Two silent failure modes are now retried instead of parsed. A residential exit that TikTok geolocates to a country it does not serve gets bounced to www.tiktok.com with HTTP 200 and no captcha marker (measured on roughly a third of Apify's Singapore exits), and www.tiktok.com serves a 1.5 KB WAF interstitial that also looks like a normal page. Both are now detected and rotate the session to a new IP rather than surfacing later as "component not found".
maxItems is respected exactly. Pages arrive in batches of 20–30, and the last batch used to stream into the dataset whole before the limit was applied — a run capped at 25 items pushed 30. On the pay-per-result plan those extra rows were also billed.
Localisation details that were wrong before: the Accept-Language header now follows the storefront, and log lines print the market's currency instead of a hardcoded $.
0.3 (2026-08-21)
Fixed category and store runs failing with SSLError: curl: (35) TLS connect error ... invalid library (0). That error is a residential exit node resetting the TLS handshake, and three network calls on the pagination path had no retry at all — a single reset on any of them killed the whole query, so a category run that had already parsed 15 products pushed 0 items. Every network call now retries (src/net.py), and a request that keeps failing raises NetworkError, which restarts the query on a new proxy IP instead of ending it.
Pagination endpoints are now called unsigned first. They answer code=0 as long as the request carries the cookies TikTok's own page handed out; the X-Tts-Oec-Bsid signer (Node.js sidecar, Bablosoft fingerprint, /bs/rt bootstrap) is spawned only if TikTok answers with code=10000/100000. A 60-product category scrape went from a ~90 s signer warmup plus a hard failure to ~13 s end to end.
Store products work again. They were fetched from an endpoint that answered code=10000 for every unverified session; unsigned requests get the real listing, and the products already rendered on the store page are used as the first page, so store products survive even when the endpoint is risk-controlled.
Items now reach the dataset as they are scraped instead of in one dump when the query ends: every SSR page, every pagination page and every store product is pushed the moment it is parsed, so a 3000-item run is usable while it is still running, and a query that dies on page 40 keeps the 39 pages that worked. Retries that re-scrape the first pages do not create duplicates — items are deduplicated by id before being pushed.
An empty product feed is now treated as a soft block instead of a result. TikTok serves the same category or search page with an empty feed on some residential exits (measured 1 in 5) — the query is retried on a fresh IP, and only if every attempt comes back empty does it report "no products found".
Bablosoft's fingerprint endpoint is now given one fast attempt per run instead of three slow ones per session — it is a paid endpoint and unreachable through most residential exits, and the bundled fingerprint is what actually gets used (this used to cost ~50 s of every query).
Input form rebuilt: the scrape type, all six input fields and the item limit are on the form itself instead of behind six collapsible sections. Only fine-tuning options and the proxy stay collapsed.
0.2 (2026-07-25)
Fixed product, reviews and store modes returning captcha errors. curl_cffi's impersonate="chrome" alias follows the newest profile in the installed library (0.15.0 → chrome146), and TikTok blocks the TLS fingerprints of recent Chrome builds — PDP and store pages answered with a captcha on every request. The profile is now pinned to chrome124 (IMPERSONATE in src/signer.py).
Reviews now come from the product page itself (review_info in the SSR payload) instead of the reviews API, which answers code=10000 for signed and unsigned requests alike. Total review count and the full star breakdown stay exact; sorting, filtering and the item limit are applied to the reviews we actually have. The API is still tried when more reviews are requested than the page carries, so extra pages are picked up automatically if TikTok reopens it.
Product mode with includeReviews no longer makes a second request or starts the signer — the reviews ship with the product page response.
Added the bundled signer/fingerprint.json fallback. Bablosoft turned its fingerprint endpoint into a paid one ({"valid":false,"message":"Key is empty"}), and the fallback pointed at a file that did not exist, so category and store runs failed outright. Sessions no longer waste retries on that endpoint either.
Target pages are now always the session's first request. A session that already carried a ttwid cookie from the warmup GET got a captcha on product and store pages.
0.1 (2026-07-07)
Six scrape modes in one Actor: search keywords, category URLs, product detail URLs, product reviews, store URLs, and creator usernames.
Search and category modes return product cards with productId, title, currentPrice, originalPrice, discountPercent, rating, reviewCount, salesVolume, sellerName, tags, imageUrls, and searchRank.
Product detail mode returns full data: description, category, price range, variants (SKUs with per-variant price and stock), specifications, selling points, brand, shipping info, seller/shop stats, and experience scores. Optionally embeds full reviews via includeReviews.
Reviews mode outputs one dataset item per review with reviewId, reviewerName, rating, text, date, variant, country, verified-purchase flag, images, and product-level ratings breakdown.
Store mode returns shop analytics (ratings, sold counts, followers, sub-scores, identity label) plus scraped store products and an estimated GMV summary.