# Changelog of TikTok Shop Scraper API — Products, Reviews, GMV | $2/1k (`pro100chok/tiktok-shop-scraper-usage`) Actor

- **URL**: https://apify.com/pro100chok/tiktok-shop-scraper-usage/changelog.md
- **Full Actor documentation**: https://apify.com/pro100chok/tiktok-shop-scraper-usage.md

## Changelog

### 0.4.12 (2026-10-01)

- **US keyword search returns the full result set again — 195 products instead of 15.** On about half of the US exit IPs TikTok answers a search with a cut-down result: the feed gives 15-20 products instead of up to 199, and every search page shows the same 10. Nothing looked wrong — the products were on topic — but a run could return a tenth of what the keyword has. The Actor now recognises that answer and asks again from a new IP (up to three times), keeping the fullest result; nothing from the cut-down answer is pushed ahead of the real one. Measured live: 195 of 195 products on three runs in a row, two of which hit a cut-down answer first.
- **Sorted searches no longer stop after the first page.** A search page that comes back empty although more results exist is re-asked from a new IP instead of ending the search.
- **Products with no reviews report `reviewCount: 0` and no rating**, instead of a text value or a fake rating of 0.

### 0.4.9 (2026-09-30)

- **Fewer products stuck at 3 reviews.** On some products TikTok challenged every second load of the product page, so the reviews session never got going and the run fell back to the reviews shown on the page (e.g. 3 of 631). The reviews session now continues on the connection that has already loaded the product page instead of loading it again, and if TikTok does put its puzzle in front of the page, the Actor solves it and carries on.

### 0.4.7 (2026-09-30)

- **Reviews: every review, not just the first three.** Products with hundreds or thousands of reviews used to come back with the 3 reviews embedded in the product page (plus the exact total and star breakdown) and a warning that TikTok only serves the rest to verified sessions. The Actor now pages through TikTok's reviews feed 20 at a time until `maxReviews` is reached or the product runs out — 2,500 reviews of a 5,600-review best-seller in about six minutes, all 1,036 reviews of a Vietnamese product in three. Works in Reviews mode and in Product mode with `includeReviews`, in all six markets.
- **How:** each page is requested the way TikTok's own storefront signs it — the security signature now covers the request body, and the request presents itself as a current Chrome. One signed session per market is prepared per run (about 40 seconds), after which each page takes a fraction of a second and about 5 KB of traffic. If TikTok answers with its slide puzzle, the Actor solves it on its own and carries on. If the feed still cannot be reached, you get the reviews from the product page as before, with a clear log line saying so — the run never fails because of it.
- **Sorting and filters now happen on TikTok's side**, so they apply to all reviews rather than to the first few: `reviewsSortBy` (`recommended` / `recent`), `reviewsStarRating` (1–5) and `reviewsFilterType` (`verified` / `with_media`). When both a star rating and a type filter are set, the star rating is applied by TikTok and the type filter on top of it. TikTok's "recommended" order reshuffles while you page through it, so for large requests the Actor tops the list up from the "recent" order — no duplicates, no gaps.
- Reviews stream into the dataset page by page while the run is going.

### 0.4.6 (2026-09-30)

- Products with no reviews now report `reviewCount` 0 and no rating, instead of a blank/text value.

### 0.4.4 (2026-09-25)

- **Product pages came back empty on every storefront — fixed.** TikTok moved the product data on its product pages (title, price, variants, rating, sales, specifications, seller) to a new place, and the Actor kept returning product rows with the title and most fields blank. The new layout is now read in full on all six markets: title, description, images, price and price range, currency, every variant with its own price, stock and picture, sales count, rating and review count, specifications and brand, shipping (delivery window, fee, free shipping), and the shop's details and scores. The previous layout is still understood, in case TikTok rolls back.
- **A product page that cannot be read now fails instead of producing a blank row**, so an empty record is never pushed or charged again.
- **Prices outside the US are real numbers.** Vietnam and other markets format prices with a thousands separator ("27.570 ₫"); prices are now taken from the raw per-variant values, so a 27,570 ₫ item is `27570`, not `27.57`. The same applies to US items over $999.
- **Fewer failed products in Malaysia, Thailand and Vietnam.** Some exits answer with an empty stub page instead of the product; that is now treated like a captcha — the Actor switches to a new IP and tries again — instead of failing the product on the first attempt.

### 0.4 (2026-09-04)

- **Six marketplaces instead of one.** A new Region dropdown — the first field on the form — covers 🇺🇸 US, 🇸🇬 Singapore, 🇲🇾 Malaysia, 🇹🇭 Thailand, 🇻🇳 Vietnam and 🇵🇭 Philippines: the storefronts TikTok publishes as hreflang whose keyword search it actually serves on the web. The UK, Japan and Mexico render fine but have no web search (the UK edge routes no search endpoint; Japan and Mexico answer every keyword with an empty result set), so they are left out rather than shipped half-working — along with Brazil (login wall for anonymous visitors), Indonesia (a different site, shop-id.tokopedia.com) and the EU markets (app-only, no web storefront at all). Category listings (with full pagination), product pages and reviews were verified live in all nine; Search and Store stay US-only because those two routes exist only on TikTok's US edge — `shop.tiktok.com/gb/s?q=...` and `/th/store/...` answer a bare 404, and so does a *US* store URL requested from a Singapore IP. Picking an unsupported mode for a region now fails immediately with a message naming the regions that do serve it, instead of scraping nothing for five minutes.
- **Keyword search reaches 6 of the 9 markets, and goes deeper everywhere.** There is no search *page* outside the US, but the feed endpoint behind it — `POST /api/shop/brandy_desktop/s/product_list` — is routed on every edge except the EU one, so search no longer needs the page. It works in the US, Singapore, Malaysia, Thailand, Vietnam and the Philippines. The keyword travels in `load_more_params.search_word`, and one call returns up to 199 products (`count=200`; anything higher answers empty, and offset/cursor/page\_token are echoed back unchanged, so 199 is the whole result set). That also lifts the US ceiling: a US keyword used to yield 20-50 products across ten server-rendered pages, and now returns up to 199 in one request. A sorted US search still takes the old page, which is the only surface that applies a sort order. Three markets have no web search and are marked as such in the input: the UK edge routes neither the search page nor the feed endpoint (404 on both, and on the suggestion endpoint and the H5 view too), and Japan and Mexico accept the request but answer with an empty result set for every keyword and every request shape tried.
- **The WAF cookies are what blocked search outside the US.** A storefront page hands out three cookies: `ttwid`, plus `dkms-token` and `dkms-type`, which carry the edge WAF's verdict on the session. On every non-US edge the search feed refuses any request that presents that pair — `code=10000`, on every exit IP, from two different proxy providers, signed or not — and serves the very same request with `ttwid` alone. Measured: full jar 0/6, ttwid only 5/5 with 140-180 products of the market's own catalogue. The search request now drops those two cookies (it has to clear the jar: curl\_cffi merges a per-request `cookies=` mapping into the session's jar rather than replacing it). US is the opposite and keeps its full jar, so the policy is per region.
- **Search is never signed.** Escalating to `X-Tts-Oec-Bsid` does not merely fail to help on this endpoint, it breaks it: in a paired test on one session the plain request returned 20 products and the signed one that followed answered `code=10000`. Twice.
- **Search calls are paced, and a throttle is waited out instead of retried into.** TikTok rate-limits the search feed per marketplace, and the limit is not per exit IP — 24 distinct residential IPs were refused together, two unrelated proxy providers behaved identically, and the same endpoint answered normally minutes later at a slower cadence. The Actor's usual answer to a rejection (drop the session, take a new IP, try again immediately, ten times over) is therefore exactly what causes it, so search now issues one call at a time a few seconds apart, waits 20 s and then 45 s in the same session if it is told to back off, and reports the keyword as throttled instead of hammering. Everything else keeps the old session-rotation behaviour, which is right for the risk control it actually faces.
- **The WAF cookies were what blocked search outside the US.** A storefront page hands out three cookies: `ttwid`, plus `dkms-token` and `dkms-type`, which carry the edge WAF's verdict on the session. Every non-US edge refuses a search request that presents that pair — `code=10000`, on every exit IP, from two different proxy providers, signed or not — and serves the very same request with `ttwid` alone. Measured: full jar 0/6, ttwid only 5/5 with 140-180 products of the market's own catalogue. The search request now clears those two out of the session jar; it has to be the jar, because curl\_cffi merges a per-request `cookies=` mapping into it rather than replacing it. The US edge is the opposite and keeps its full jar, so the policy is per region.
- **Search is never signed.** Escalating to `X-Tts-Oec-Bsid` does not merely fail to help on this endpoint, it breaks it: in a paired test on a single session the plain request returned 20 products and the signed one that followed answered `code=10000`. Twice over.
- **The request signer now speaks all three of TikTok's security stacks.** TikTok runs separate web-security services per edge — `ttp` for the US, `sg` for SEA/JP/MX, `eu` for the UK — each with its own unisec loader, core, bssdk and webmssdk bundle and its own msToken host. The sidecar had the US bundles hardcoded, so every signed request made for a non-US storefront carried a signature the wrong service would never accept. All three stacks are now vendored under `signer/sdk/<stack>/` and the sidecar is spawned on the one belonging to the region, with the mocked page pointing at that storefront. This is what makes escalation meaningful outside the US at all.
- **The proxy field is gone.** TikTok blocks datacenter IPs and serves each storefront only to its own country, which left exactly one workable setting — so the Actor now picks it: Apify Residential in the region's country, one configuration per country, a fresh IP per session. Existing runs that still pass `proxyConfiguration` keep working; the value is simply ignored.
- **A pasted URL now decides its own market.** `https://shop.tiktok.com/gb/store/...` is scraped from UK IPs even if the dropdown was left on United States, and a run may mix markets — each query gets the right country. A URL naming a market the Actor cannot reach is rejected up front with the list of supported ones.
- **Creator scrapes always run on the US pool.** Creator profiles come from `www.tiktok.com`, which answers most non-US residential exits with a WAF challenge instead of the profile (measured 6/6 on Japanese, Mexican and SEA exits, 0/5 on US). That one mode is now pinned to US IPs regardless of the selected region, so it keeps working while the rest of the run stays in its market.
- **Every row says which market it came from.** New `storefront` (`us`, `gb`, `th`, …) and `currency` fields on every product, review and store record, and as columns in the dataset views — prices arrive as bare numbers, so a dataset mixing markets was previously ambiguous. Creator rows are left alone: they have no market and no prices.
- **Session warmup is now a per-market setting.** The right behaviour genuinely differs: on the UK storefront a cold product page hits "Security Check" on about two thirds of products and loads reliably once the session has visited `/gb`, while on Thailand the reverse is true (a warmed session captcha'd 8/8, cold passed 3/3). Each region carries the measured answer instead of one global rule.
- **Two silent failure modes are now retried instead of parsed.** A residential exit that TikTok geolocates to a country it does not serve gets bounced to `www.tiktok.com` with HTTP 200 and no captcha marker (measured on roughly a third of Apify's Singapore exits), and `www.tiktok.com` serves a 1.5 KB WAF interstitial that also looks like a normal page. Both are now detected and rotate the session to a new IP rather than surfacing later as "component not found".
- **`maxItems` is respected exactly.** Pages arrive in batches of 20–30, and the last batch used to stream into the dataset whole before the limit was applied — a run capped at 25 items pushed 30. On the pay-per-result plan those extra rows were also billed.
- Localisation details that were wrong before: the `Accept-Language` header now follows the storefront, and log lines print the market's currency instead of a hardcoded `$`.

### 0.3 (2026-08-21)

- Fixed category and store runs failing with `SSLError: curl: (35) TLS connect error ... invalid library (0)`. That error is a residential exit node resetting the TLS handshake, and three network calls on the pagination path had no retry at all — a single reset on any of them killed the whole query, so a category run that had already parsed 15 products pushed 0 items. Every network call now retries (src/net.py), and a request that keeps failing raises `NetworkError`, which restarts the query on a new proxy IP instead of ending it.
- Pagination endpoints are now called unsigned first. They answer `code=0` as long as the request carries the cookies TikTok's own page handed out; the `X-Tts-Oec-Bsid` signer (Node.js sidecar, Bablosoft fingerprint, `/bs/rt` bootstrap) is spawned only if TikTok answers with `code=10000`/`100000`. A 60-product category scrape went from a ~90 s signer warmup plus a hard failure to ~13 s end to end.
- Store products work again. They were fetched from an endpoint that answered `code=10000` for every unverified session; unsigned requests get the real listing, and the products already rendered on the store page are used as the first page, so store products survive even when the endpoint is risk-controlled.
- Items now reach the dataset as they are scraped instead of in one dump when the query ends: every SSR page, every pagination page and every store product is pushed the moment it is parsed, so a 3000-item run is usable while it is still running, and a query that dies on page 40 keeps the 39 pages that worked. Retries that re-scrape the first pages do not create duplicates — items are deduplicated by id before being pushed.
- An empty product feed is now treated as a soft block instead of a result. TikTok serves the same category or search page with an empty feed on some residential exits (measured 1 in 5) — the query is retried on a fresh IP, and only if every attempt comes back empty does it report "no products found".
- Bablosoft's fingerprint endpoint is now given one fast attempt per run instead of three slow ones per session — it is a paid endpoint and unreachable through most residential exits, and the bundled fingerprint is what actually gets used (this used to cost ~50 s of every query).
- Input form rebuilt: the scrape type, all six input fields and the item limit are on the form itself instead of behind six collapsible sections. Only fine-tuning options and the proxy stay collapsed.

### 0.2 (2026-07-25)

- Fixed product, reviews and store modes returning captcha errors. curl\_cffi's `impersonate="chrome"` alias follows the newest profile in the installed library (0.15.0 → chrome146), and TikTok blocks the TLS fingerprints of recent Chrome builds — PDP and store pages answered with a captcha on every request. The profile is now pinned to chrome124 (`IMPERSONATE` in src/signer.py).
- Reviews now come from the product page itself (`review_info` in the SSR payload) instead of the reviews API, which answers `code=10000` for signed and unsigned requests alike. Total review count and the full star breakdown stay exact; sorting, filtering and the item limit are applied to the reviews we actually have. The API is still tried when more reviews are requested than the page carries, so extra pages are picked up automatically if TikTok reopens it.
- Product mode with `includeReviews` no longer makes a second request or starts the signer — the reviews ship with the product page response.
- Added the bundled `signer/fingerprint.json` fallback. Bablosoft turned its fingerprint endpoint into a paid one (`{"valid":false,"message":"Key is empty"}`), and the fallback pointed at a file that did not exist, so category and store runs failed outright. Sessions no longer waste retries on that endpoint either.
- Target pages are now always the session's first request. A session that already carried a `ttwid` cookie from the warmup GET got a captcha on product and store pages.

### 0.1 (2026-07-07)

- Six scrape modes in one Actor: search keywords, category URLs, product detail URLs, product reviews, store URLs, and creator usernames.
- Search and category modes return product cards with productId, title, currentPrice, originalPrice, discountPercent, rating, reviewCount, salesVolume, sellerName, tags, imageUrls, and searchRank.
- Product detail mode returns full data: description, category, price range, variants (SKUs with per-variant price and stock), specifications, selling points, brand, shipping info, seller/shop stats, and experience scores. Optionally embeds full reviews via includeReviews.
- Reviews mode outputs one dataset item per review with reviewId, reviewerName, rating, text, date, variant, country, verified-purchase flag, images, and product-level ratings breakdown.
- Store mode returns shop analytics (ratings, sold counts, followers, sub-scores, identity label) plus scraped store products and an estimated GMV summary.
- Creator mode returns profile data: username, nickname, bio, region, verified/seller flags, and follower/following/like/video counts.
- Residential (US) proxy support with automatic session rotation and retry on captcha or HTTP errors.
- Dataset views for Products, Product Details, Stores, Store Products, Reviews, and Creators.