# Changelog of TikTok Ad Library Scraper: Ads, Targeting & Spend (`rastriq/tiktok-ad-library-scraper`) Actor

- **URL**: https://apify.com/rastriq/tiktok-ad-library-scraper/changelog.md
- **Full Actor documentation**: https://apify.com/rastriq/tiktok-ad-library-scraper.md

## Changelog — TikTok Ad Library Scraper

### \[Unreleased] — 2026-11-10

- Advertiser-name searches now say how the name was resolved. New output field `advertiserMatch` on every ad: `exact`, `closest_suggestion` (no exact name: TikTok's first suggestion, possibly another business) or `keyword_fallback` (no advertiser found: keyword search, may include other advertisers); `null` for keyword and ID searches. Non-exact matches add a note to the status message and the resolution (with TikTok's top suggestions) to `RUN_SUMMARY.advertiserMatch`.
- New optional input `strictAdvertiserMatch` (default `false`): returns 0 results instead of falling back when no advertiser has exactly that name.
- Exact-name comparison now ignores surrounding spaces. Default behaviour (which advertiser is picked, keyword fallback) is unchanged.

### \[vis-2026-11] — 2026-11-09

- README/SEO only, no code change: added "TikTok Ads Library" variant to the Store text, new "Use with Claude/ChatGPT (MCP)" section, reworded the sample-run line in the input schema description. Input schema prefill values unchanged.

### \[0.3.7] — 2026-09-14

#### Fixed — HTTP 421 "system busy" definitively resolved

The core 421 error that blocked all POST `/api/v1/search` requests has been
eliminated. The root cause was that TikTok's `/search` endpoint requires
cryptographic request signing (Argus/webmssdk) and a valid `msToken` cookie,
neither of which could be generated in a headless Playwright context. The
`/suggestion` endpoint (autocomplete) worked because TikTok does not enforce
signing on lower-risk endpoints.

**New approach — sticky-session real navigation:**

Instead of crafting signed POST requests, the actor now:

1. Captures `x-ccl-str` autonomously from the live browser session (no manual
   injection required)
2. Fetches ad detail pages via real in-browser navigation within the same
   sticky session, inheriting all signing, cookies, and TLS context
3. Avoids the 421 entirely by never issuing a bare programmatic POST to
   `/search` outside the SPA's own request pipeline

#### Fixed — Commercial content pagination

The cursor for `/other-commercial-contents/search` was advancing by 1 instead
of by element count, causing the pagination to stall after the first page.
Fixed to advance by the actual number of elements returned per page (36).

#### Fixed — Deduplication

Added robust deduplication using `contentKey`. When TikTok omits the `id` field
from commercial content elements, the actor now generates a deterministic
`contentKey` to prevent duplicate or lost records.

#### Added — Enriched detail mode

`includeDetails=true` now returns up to 49 fields per ad, including: public
advertiser identity, destination URL and CTA, ad objective, public TikTok
profile, and targeting breakdowns (location, age, gender, countries, cities,
languages, devices, operating systems, interests, and interactions) when
provided by TikTok.

#### Added — Output schema improvements

- `recordType` discriminator field across all output types
- `scrapedAt` ISO-8601 timestamp on every record
- Full `rawData` payload preserved for downstream analysis
- `dataset_schema.json` and `output_schema.json` expanded with types,
  descriptions, and five Apify Console display views (Overview, Ads, Targeting,
  Commercial content, Raw data)

#### Fixed — Docker build

`.actor` directory now explicitly copied into the Docker image, fixing metadata
resolution at runtime.

#### Validation results (beta 0.3.7)

| Test | Run ID | Result |
|------|--------|--------|
| Search + pagination (100 ads) | `ExCfGbIljnWFMKZOy` | SUCCEEDED — 100 unique IDs, 23 fields |
| Enriched detail (2 ads) | `Xu5hryG4bqYPRNjDo` | SUCCEEDED — 2 details, 49 fields |
| Commercial content (40 records) | `i7kJcGxNlMOdCPEyR` | SUCCEEDED — 40 unique keys, 18 fields |
| Advertiser discovery (3 advertisers) | `XdGOHkhNAjBD63ZUI` | SUCCEEDED — 3 unique IDs, 5 fields |

#### Investigation context (builds 0.2.9–0.2.13)

The 421 investigation spanned builds 0.2.9 through 0.2.13 across 6 test runs.
Key findings: SPA loads and is functional (POST `/suggestion` works from the
React app), but `msToken` never generates in headless mode and Argus/webmssdk
signing APIs are encapsulated as internal webpack modules (not exposed as
`window` globals). An in-page `fetch()` to `/search` from the browser context
also returned 421, confirming the block is at the request-signing layer, not
TLS. Full investigation report: `INFORME_INVESTIGACION_421.md`.

***

### \[0.1.40] — 2026-08-22

#### Changed — Switched from curl_cffi to Playwright browser (BREAKING)

**Root cause confirmed:** TikTok's Ad Library CDN rejects ALL non-browser TLS
fingerprints with `421 "system busy"`. This was proven definitively on
2026-08-22 by running the exact browser `curl` command (same headers, cookies,
body, X-CCL-STR) from `curl.exe` on the user's own ISP IP — still 421. The
browser on the same machine/IP succeeds. The block is purely TLS/JA4
fingerprint-based, not IP-based.

**What changed:**

- Replaced `curl_cffi` HTTP client with **Playwright Chromium** browser
- All API calls (`/search`, `/suggestion`, `/items/*/details`,
  `/other-commercial-contents/search`) now execute via `page.evaluate(fetch())`
  inside a real browser context
- The browser navigates to `library.tiktok.com/ads` first to establish a
  legitimate session (cookies: `ttwid`, `msToken`, `odin_tt`, `tt_chain_token`)
- `X-CCL-STR` token obtained via browser's own `fetch('/api/v1/support-regions')`
- On 421 retry, the browser reloads the page for a fresh session

**Dependency changes:**

| Before | After |
|--------|-------|
| `curl_cffi>=0.16.0` | `playwright>=1.48.0` |

**Dockerfile changes:**

- Added `python -m playwright install --with-deps chromium`
- Chromium launched with `--no-sandbox --disable-dev-shm-usage` (required in Docker)

**Runtime requirements:**

- Minimum memory increased from 256MB to **2048MB** (Chromium browser overhead)
- Default `memoryMbytes` in `actor.json` updated accordingly

**Diagnostic findings documented:**

| Test | Result |
|------|--------|
| curl_cffi + impersonate="chrome136" from ISP IP | 421 |
| curl.exe (SChannel TLS) from ISP IP | 421 |
| Browser (Firefox 154.0) from same ISP IP | 200 |
| Cookies (all 4 combinations ± cookies) | No effect on 421 |
| Decodo residential (ASN-12430, ASN-3352) | 421 |
| Apify RESIDENTIAL proxy + Playwright | 200 |

***

### \[0.1.30] — 2026-08-22

#### Fixed — API body format update (based on JS bundle reverse engineering)

TikTok's `/api/v1/search` endpoint now validates the full set of filter fields
that the browser frontend always sends. Requests missing these fields received
`421 "system busy"` at the application layer (after passing the TLS check).

**New required body fields added to `search_ads()`:**

| Field | Type | Default | Notes |
|-------|------|---------|-------|
| `ad_type` | int | `0` | 0 = all types (video/image/text) |
| `ad_status` | int | `0` | 0 = all statuses (active + inactive) |
| `ages` | list | `["all"]` | Array, not string — separator is `;` in the browser |
| `ad_reach` | list | `["all"]` | Array, not string |
| `gender` | str | `"ALL"` | ALL / FEMALE / MALE |

Values sourced directly from the TikTok Ad Library JS bundle
(`main.9fa590f3.js`, function `ec()`/`eu()` + filter definitions).

**Exact-match (quoted) search support added:**

When `searchQuery` is wrapped in double quotes (e.g. `"nike"`), the actor now
uses the `search_clause` body path instead of the plain `query` path, exactly
mirroring what the browser frontend does:

```json
// Quoted ("nike") → search_clause path
{ "search_clause": { "search_terms": ["nike"], "search_type": 1 } }

// Plain (nike) → query path
{ "query": "nike" }
```

**Improved diagnostics for 421 errors:**

- Proxy host:port now logged at session init (credentials never logged)
- TLS impersonate profile logged at session init
- 421 response body included in the warning to distinguish
  `"system busy"` (edge/IP rejection) from `"params error"` (API validation)

#### Known limitation — IP-quality requirement

TikTok's Ad Library POST endpoints (`/search`, `/suggestion`) reject requests
from datacenter and known-proxy IP ranges with `421 "system busy"`. This is
enforced at the CDN edge layer **before** TLS fingerprint or body validation.

Confirmed blocked (all return `421 "system busy"`):

- Apify datacenter proxies
- Apify RESIDENTIAL proxy group
- Decodo residential (`es.decodo.com:10001`)

Confirmed working:

- Direct ISP residential connection (verified with `curl.exe` on Windows from
  a Spanish ISP, no proxy)

**Workaround:** configure a truly ISP-native residential proxy whose IP range
is not in TikTok's CDN block-list. The actor's proxy configuration accepts any
HTTP proxy URL via `proxyUrls` in `proxyConfiguration`.

***

### \[0.1.28] — 2026-08-22

#### Fixed

- **421 Misdirected Request on all POST endpoints** — Root cause: the profile list
  `["chrome", "chrome131", "chrome120", "safari17_0"]` was entirely rejected by
  TikTok's JA3/JA4 TLS fingerprint allow-list.
  - `"chrome"` is **not** a valid `curl_cffi` profile name (causes silent fallback
    to a generic TLS stack that fails immediately).
  - `chrome120`, `chrome131`, and `safari17_0` are now on TikTok's block-list as
    of mid-2026.
- Updated `IMPERSONATE_PROFILES` to use current, versioned profiles available in
  `curl_cffi>=0.16.0`:
  ```
  chrome136, chrome142, chrome146, chrome150,
  firefox144, firefox147, safari18_0, safari184
  ```
- Raised default `max_retries` in `_request()` from 3 → 5 so the actor cycles
  through more fingerprints before aborting.
- Bumped `curl_cffi` minimum requirement from `>=0.7.0` to `>=0.16.0` to ensure
  the new profiles are always available.

### \[0.1.0] — 2026-06-30

#### Added

- Initial release: `search_ads`, `discover_advertisers`, `search_commercial_content` modes.
- URL-based input from TikTok Ad Library share links.
- `curl_cffi` TLS impersonation to bypass JA3/JA4 fingerprint gating.
- Proxy support via Apify proxy configuration.
