# Douyin Scraper — Hot Trends, Videos & TikTok China Data (`bovi/douyin-scraper`) Actor Scrape Douyin (抖音) real-time trending topics with hot\_value, view\_count, discuss\_count. Also supports user\_videos and video\_detail modes. Auto-registers ttwid — no manual cookies needed. a\_bogus signed, CN-residential proxy. - **URL**: https://apify.com/bovi/douyin-scraper.md - **Developed by:** [Vitalii Bondarev](https://apify.com/bovi) (community) - **Categories:** Social media, Marketing - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing from $0.48 / 1,000 hot search topics This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Douyin Scraper — Video Search, User Posts & Comments **Douyin Scraper** lets you scrape Douyin (抖音) videos, user posts, comments, and full video metadata. Returns 20+ fields per video including play count, digg count (likes), video URL, cover image, music info, and hashtags. Works without a browser — uses a pure-Python a_bogus request signer (SM3 + RC4 + custom Base64) instead of Playwright overhead. Four scrape modes in one actor: keyword search, user videos, video comments, and video detail lookup. **Requires CN-residential proxy** — Douyin blocks non-China IPs. ### Douyin Scraper Modes | Mode | Input | What You Get | |------|-------|--------| | `search` | keywords | Videos matching keyword search | | `user_videos` | sec_user_ids | All public videos from a Douyin user | | `comments` | aweme_ids | Comments on a specific video | | `video_detail` | aweme_ids | Full metadata for specific video IDs | ### Douyin Video Data Fields Each video record includes 20+ fields: - `aweme_id` — Douyin video ID - `desc` — video description / caption - `author_uid`, `author_nickname`, `author_sec_uid` — author info - `digg_count` — total likes - `comment_count` — comment count - `share_count` — share count - `play_count` — total play count - `collect_count` — save/collect count - `video_url` — direct MP4 video file URL - `cover_url` — cover image URL - `music_title`, `music_author`, `music_url` — background music info - `create_time` — ISO-8601 UTC timestamp - `duration_ms` — video duration in milliseconds - `is_top` — whether the video is pinned on user profile - `hashtags` — list of hashtag strings - `parse_confidence` — data completeness score (0.0–1.0) - `warnings` — list of missing or degraded fields - `scraped_at` — scrape timestamp **Comment records** include: `comment_id`, `aweme_id`, `text`, `author_uid`, `author_nickname`, `digg_count`, `reply_count`, `create_time`. ### How to Use Douyin Scraper #### Search Douyin Videos by Keyword ```json { "mode": "search", "keywords": ["美食", "穿搭"], "maxItems": 100, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"], "apifyProxyCountry": "CN" } } ```` #### Scrape Douyin User Videos by sec\_uid ```json { "mode": "user_videos", "secUserIds": ["MS4wLjABAAAA..."], "maxItems": 200, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"], "apifyProxyCountry": "CN" } } ``` #### Scrape Douyin Comments on a Video ```json { "mode": "comments", "awemeIds": ["7123456789012345678"], "maxItems": 500 } ``` ### Proxy Requirement — Why CN-Residential Is Needed **CN-residential proxy is required.** Douyin aggressively blocks: - Datacenter IPs (Apify cloud default) - Non-China IP addresses - Suspicious TLS fingerprints Configure `proxyConfiguration` with `countryCode: CN` and group `RESIDENTIAL`. Without it, most requests will get captcha'd or blocked. Proxy cost is billed to your Apify account at standard residential rates. ### Optional Cookies for Higher Success Rate Douyin uses `msToken`, `ttwid`, and `s_v_web_id` cookies for device fingerprinting. You can supply real cookies from a browser session: 1. Open `www.douyin.com` in Chrome 2. Open DevTools → Application → Cookies → `www.douyin.com` 3. Copy `msToken`, `ttwid`, `s_v_web_id` values into actor input Fake cookies are auto-generated if not supplied, but real cookies improve reliability for sustained scraping. ### Why This Douyin Scraper Beats Competitors | | This actor | zen-studio/douyin-search-scraper | |---|:---:|:---:| | Price | $2.00/1000 | $4.99/1000 | | Modes | 4 (search + user + comments + detail) | search only | | Browser needed | No (pure-Python signer) | Yes (Playwright) | | Comments support | Yes | No | | parse\_confidence signal | Yes | No | ### Pricing — Pay Per Douyin Video or Comment **$2.00 per 1,000 results** (PPE — pay per result). CN-residential proxy cost is billed separately at standard Apify residential rates. ### Technical Notes - **a\_bogus signer**: pure Python, no browser. Algorithm: SM3 hash (via `gmssl`) → RC4 → custom Base64 (s4 alphabet). Vendored from Evil0ctal/Douyin\_TikTok\_Download\_API (Apache 2.0). - **Reliable HTTP access**: lightweight requests pass Douyin's network checks. - **Maintenance**: Douyin rotates the a\_bogus signing algorithm roughly quarterly. Signer is monitored and patched. ### Limitations - Private accounts and private videos require login (not supported) - CN-residential proxy is buyer's cost (standard Apify residential pricing) - Video URLs from Douyin CDN expire within hours to days — download promptly if needed *** *Not affiliated with Douyin / ByteDance. Data retrieved from publicly accessible Douyin endpoints.* ### Integrations Built for social-listening teams and content-intelligence analysts tracking video trends, engagement metrics, and comments on Douyin — the JSON/dataset output drops into the tools you already run, no glue code: - **n8n / Make / Zapier** — trigger a run or pipe every new dataset item into 500+ apps (Google Sheets, Airtable, Slack, HubSpot, your database) with no code: [n8n](https://docs.apify.com/platform/integrations/n8n), [Make](https://docs.apify.com/platform/integrations/make), [Zapier](https://docs.apify.com/platform/integrations/zapier). - **Webhooks** — fire your own endpoint the moment a run finishes, to push results straight into your pipeline ([docs](https://docs.apify.com/platform/integrations/webhooks)). - **MCP server** — expose this actor as a tool to Claude, Cursor, or any [MCP client](https://mcp.apify.com) so an AI agent can pull this data mid-conversation ([guide](https://blog.apify.com/how-to-use-mcp/)). - **API & SDKs** — fetch the dataset as JSON, CSV, or Excel through the Apify REST API or the Python / JS SDKs. See all [Apify integrations](https://apify.com/integrations). # Actor input Schema ## `mode` (type: `string`): Scraping mode. 'hot\_search' = Douyin trending topics (PUBLIC, no login). 'search'/'user\_videos'/'comments'/'video\_detail' require real login cookies (degraded as of 2026). ## `keywords` (type: `array`): Keywords to search for. Used in mode=search. Example: \['美食', '探店', '旅游']. Each keyword runs a separate search up to maxItems results. ## `sec_user_ids` (type: `array`): Douyin sec\_user\_id strings for user video mode. The sec\_user\_id is found in the Douyin user profile URL or API responses. Example: \['MS4wLjABAAAA...']. ## `aweme_ids` (type: `array`): Douyin video IDs (aweme\_id) for comments or video\_detail mode. Example: \['7345492945006595379']. Find these in video URLs or previous search results. ## `maxItems` (type: `integer`): Maximum number of videos to return per keyword or user. Default: 50. In comments mode, use maxComments instead. ## `maxComments` (type: `integer`): Maximum number of comments to return per video (mode=comments only). Default: 50. ## `msToken` (type: `string`): Optional: Douyin msToken cookie value from a real browser session. Improves success rate significantly. Get it from browser DevTools → Application → Cookies → www.douyin.com. Valid for days to weeks. ## `ttwid` (type: `string`): Optional: Douyin ttwid cookie value from a real browser session (same source as msToken). Format: '1|hash|timestamp'. ## `svWebId` (type: `string`): Optional: Douyin s\_v\_web\_id cookie (device ID). Format: 'verify\_XXXXXXXXXXXXXXXX'. ## `proxyConfiguration` (type: `object`): REQUIRED for reliable access. Douyin blocks datacenter IPs. Use Apify RESIDENTIAL proxy with countryCode=CN. Without proxy, most requests will be blocked or captcha'd. ## Actor input object example ```json { "mode": "hot_search", "keywords": [ "美食" ], "maxItems": 50, "maxComments": 50, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "CN" } } ``` # Actor output Schema ## `results` (type: `string`): Dataset containing Douyin Scraper records (rank, word, hot\_value, view\_count, video\_count, discuss\_video\_count, event\_time, sentence\_id, scraped\_at, parse\_confidence). # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "mode": "hot_search", "keywords": [ "美食" ], "maxItems": 50, "maxComments": 50, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "CN" } }; // Run the Actor and wait for it to finish const run = await client.actor("bovi/douyin-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "mode": "hot_search", "keywords": ["美食"], "maxItems": 50, "maxComments": 50, "proxyConfiguration": { "useApifyProxy": True, "apifyProxyGroups": ["RESIDENTIAL"], "apifyProxyCountry": "CN", }, } # Run the Actor and wait for it to finish run = client.actor("bovi/douyin-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "mode": "hot_search", "keywords": [ "美食" ], "maxItems": 50, "maxComments": 50, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "CN" } }' | apify call bovi/douyin-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=bovi/douyin-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Douyin Scraper — Hot Trends, Videos & TikTok China Data", "description": "Scrape Douyin (抖音) real-time trending topics with hot_value, view_count, discuss_count. Also supports user_videos and video_detail modes. Auto-registers ttwid — no manual cookies needed. a_bogus signed, CN-residential proxy.", "version": "0.1", "x-build-id": "QbxdK6yKNJMFI7PRm" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/bovi~douyin-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-bovi-douyin-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/bovi~douyin-scraper/runs": { "post": { "operationId": "runs-sync-bovi-douyin-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/bovi~douyin-scraper/run-sync": { "post": { "operationId": "run-sync-bovi-douyin-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "mode": { "title": "Mode", "enum": [ "hot_search", "search", "user_videos", "comments", "video_detail" ], "type": "string", "description": "Scraping mode. 'hot_search' = Douyin trending topics (PUBLIC, no login). 'search'/'user_videos'/'comments'/'video_detail' require real login cookies (degraded as of 2026).", "default": "hot_search" }, "keywords": { "title": "Search Keywords", "type": "array", "description": "Keywords to search for. Used in mode=search. Example: ['美食', '探店', '旅游']. Each keyword runs a separate search up to maxItems results.", "items": { "type": "string" } }, "sec_user_ids": { "title": "User sec_user_ids", "type": "array", "description": "Douyin sec_user_id strings for user video mode. The sec_user_id is found in the Douyin user profile URL or API responses. Example: ['MS4wLjABAAAA...'].", "items": { "type": "string" } }, "aweme_ids": { "title": "Video aweme_ids", "type": "array", "description": "Douyin video IDs (aweme_id) for comments or video_detail mode. Example: ['7345492945006595379']. Find these in video URLs or previous search results.", "items": { "type": "string" } }, "maxItems": { "title": "Max Items", "minimum": 1, "maximum": 10000, "type": "integer", "description": "Maximum number of videos to return per keyword or user. Default: 50. In comments mode, use maxComments instead.", "default": 50 }, "maxComments": { "title": "Max Comments", "minimum": 1, "maximum": 5000, "type": "integer", "description": "Maximum number of comments to return per video (mode=comments only). Default: 50.", "default": 50 }, "msToken": { "title": "msToken Cookie", "type": "string", "description": "Optional: Douyin msToken cookie value from a real browser session. Improves success rate significantly. Get it from browser DevTools → Application → Cookies → www.douyin.com. Valid for days to weeks." }, "ttwid": { "title": "ttwid Cookie", "type": "string", "description": "Optional: Douyin ttwid cookie value from a real browser session (same source as msToken). Format: '1|hash|timestamp'." }, "svWebId": { "title": "s_v_web_id Cookie", "type": "string", "description": "Optional: Douyin s_v_web_id cookie (device ID). Format: 'verify_XXXXXXXXXXXXXXXX'." }, "proxyConfiguration": { "title": "Proxy Configuration", "type": "object", "description": "REQUIRED for reliable access. Douyin blocks datacenter IPs. Use Apify RESIDENTIAL proxy with countryCode=CN. Without proxy, most requests will be blocked or captcha'd." } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```