# TikTok Hashtag, Search & Profile Scraper (`dtrungtin/tiktok-hashtag-search-profile-scraper`) Actor Extract TikTok videos from any hashtag page without dealing with TikTok's anti-bot defenses, sign-in walls, or undocumented APIs. - **URL**: https://apify.com/dtrungtin/tiktok-hashtag-search-profile-scraper.md - **Developed by:** [Tin](https://apify.com/dtrungtin) (community) - **Categories:** Videos - **Stats:** 7 total users, 3 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing from $10.00 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## TikTok Hashtag, Search & Profile Scraper Extract TikTok videos from any hashtag page, search query, or user profile without dealing with TikTok's anti-bot defenses, sign-in walls, or undocumented APIs. Paste one or more hashtag, search, or profile URLs and get back a structured dataset of videos with full metadata: captions, authors, play counts, likes, music, hashtags, and direct video URLs. Download your data as JSON, CSV, Excel, HTML, or XML directly from the Apify console. ### What you can do with the data - **Trend discovery** — track which videos are gaining traction inside any hashtag, who their creators are, and how engagement (plays, likes, shares) scales over time. - **Creator research** — identify rising creators inside a niche by sorting hashtag results by play count or engagement ratio. - **Music intelligence** — see which tracks are driving views inside a hashtag; useful for music labels, artist managers, and ad targeting. - **Competitive monitoring** — run scheduled scrapes of hashtags relevant to your brand or competitor campaigns and watch the leaderboard shift. - **Dataset building** — collect labeled video data (caption, hashtags, music, stats) for ML training or content classification. - **Audience research** — opt-in to `commentsPerPost` to pull comment text + commenter handles alongside each video, useful for sentiment analysis, FAQ mining, or surfacing high-intent viewers in a niche. ### Input | Field | Type | Required | Description | |---|---|---|---| | `startUrls` | array | yes | TikTok hashtag URLs (`https://www.tiktok.com/tag/`), search URLs (`https://www.tiktok.com/search?q=`), or profile URLs (`https://www.tiktok.com/@`). Individual video URLs are rejected at startup. | | `maxItems` | integer | yes | Max videos to collect per URL (default: 10, range: 1–500). | | `commentsPerPost` | integer | no | How many top-level comments to fetch per video, embedded into the video record under `comments`. Default `0` (skip comments). Range: 0–500. Enabling this navigates to each video's page individually — a value of 50 across 50 videos can add several minutes to the run. | | `countryCode` | string | no | Exit-node country for the Apify residential proxy. One of `US`, `DE`, `VN`, `FR`, `GB`. Default: `US`. Choose a country close to where the audience is concentrated for the most relevant results. | #### Example input ```json { "startUrls": [ { "url": "https://www.tiktok.com/tag/funny" }, { "url": "https://www.tiktok.com/search?q=cooking%20tips" }, { "url": "https://www.tiktok.com/@tranthanh123" } ], "maxItems": 50, "commentsPerPost": 20, "countryCode": "US" } ```` ### Output Each scraped video is pushed to the dataset as one record. Multiple URLs run in parallel; the `sourceType`, `sourceHashtag`, `sourceQuery`, `sourceUsername`, and `sourceUrl` fields let you tell records apart downstream. #### Example output record ```json { "sourceType": "hashtag", "sourceHashtag": "funny", "sourceQuery": null, "sourceUsername": null, "sourceUrl": "https://www.tiktok.com/tag/funny", "id": "7596344392374455570", "caption": "Life With Two Cats 😂 | Part 111 #cat #catsoftiktok #funnyvideos #funnycat #funnycats", "createTime": 1768661758, "url": "https://www.tiktok.com/@funnyanimalshub/video/7596344392374455570", "author": { "id": "7511384928216138808", "uniqueId": "funnyanimalshub", "nickname": "Funny Animals hub", "avatar": "https://p16-common-sign.tiktokcdn.com/...jpeg", "verified": false }, "stats": { "playCount": 23100000, "diggCount": 3200000, "commentCount": 4852, "shareCount": 606600, "collectCount": 197600 }, "video": { "duration": 65, "width": 576, "height": 1024, "cover": "https://p16-common-sign.tiktokcdn.com/...image", "dynamicCover": "https://p16-common-sign.tiktokcdn.com/...image", "playUrl": "https://v16-webapp-prime.tiktok.com/video/...", "downloadUrl": "https://v16-webapp-prime.tiktok.com/video/..." }, "music": { "id": "7596344513577388807", "title": "som original", "authorName": "Funny Animals hub", "original": true, "playUrl": "https://v16m.tiktokcdn.com/..." }, "hashtags": ["cat", "catsoftiktok", "funnyvideos", "funnycat", "funnycats"], "comments": [ { "commentId": "7640259390033167112", "text": "this cat is a whole mood", "createTime": 1778886530, "likeCount": 12, "replyCount": 0, "user": { "id": "7298733093128307714", "uniqueId": "zaii.bg", "nickname": "kunn 🐍", "avatar": "https://p16-common-sign.tiktokcdn.com/...jpg" } } ] } ``` The `comments` array is only present when `commentsPerPost > 0`. When it's `0` (the default), the field is omitted. #### Field reference - **`sourceType` / `sourceHashtag` / `sourceQuery` / `sourceUsername` / `sourceUrl`** — which input URL produced this record (set per item so concurrent scrapes don't get jumbled). `sourceType` is `"hashtag"`, `"search"`, or `"profile"`; `sourceHashtag` is populated only for hashtag URLs, `sourceQuery` only for search URLs, `sourceUsername` only for profile URLs. - **`id` / `url`** — TikTok video ID and canonical web URL. - **`caption` / `createTime`** — raw caption text and Unix timestamp of upload. - **`author`** — uploader's TikTok ID, `@handle` (`uniqueId`), display nickname, avatar, and verified flag. - **`stats`** — play / like (`diggCount`) / comment / share / save (`collectCount`) counts at scrape time. - **`video`** — duration in seconds, dimensions, static cover, animated cover, and the watermarked play URL plus the no-watermark `downloadUrl`. **Note: the play/download URLs are signed and expire within a few hours** — re-scrape or download immediately if you need the bytes. - **`music`** — sound metadata, including whether it's an original sound and a direct play URL. - **`hashtags`** — list of hashtags parsed from the caption. - **`comments`** — (optional, only when `commentsPerPost > 0`) top-level comments fetched by visiting each video's page. Each entry has `commentId`, `text`, `createTime`, `likeCount`, `replyCount`, and `user` (`id`, `uniqueId`, `nickname`, `avatar`). Reply threads are not included; only the first-level comments. ### How it works This actor uses `PuppeteerCrawler` (Crawlee) with a Chromium browser, the `puppeteer-extra-plugin-stealth` patch set, and Apify fingerprint generation to mimic a real desktop browser. Per request it: 1. Loads `tiktok.com` first as a "warm-up" — drifting the mouse, scrolling, and waiting a few seconds so that session cookies and the `msToken` settle. Cold sessions hitting `/tag/` or `/search` directly get the well-known *empty 200* soft-block. 2. Navigates to the target page and intercepts the relevant AJAX response — `/api/challenge/item_list/` for hashtag pages, `/api/search/general/full/` (or `/api/search/item/full/`) for search pages, `/api/post/item_list/` for profile pages — to extract the structured video list. Search responses are unwrapped (`data[].item`) so the downstream output schema matches the other flows. 3. Scrolls to load more videos until either `maxItems` is reached or the API reports no more results. 4. (Optional) If `commentsPerPost > 0`, navigates the same browser session to each collected video's page, intercepts `/api/comment/list/`, scrolls the comment panel, and embeds the captured comments into that video's record before pushing to the dataset. Heavy resources (images, media, fonts, stylesheets) and tracking hosts (analytics endpoints, GA, Facebook, DoubleClick) are blocked at the request level to cut bandwidth and speed up the run. ### Notes and limitations - **Use a residential proxy in production.** TikTok soft-blocks fresh datacenter IPs and IPs with scraper history by returning HTTP 200 with an empty body. The actor will log `TikTok API non-200` or `Failed to parse TikTok API JSON` when this happens — switching to `RESIDENTIAL` proxy almost always fixes it. - **Hashtag, search, and profile URLs only.** Single-video URLs (`/@user/video/123…`) are rejected at startup. Use a different actor for those. - **Signed media URLs expire.** `playUrl`, `downloadUrl`, `cover`, and `avatar` URLs are signed by TikTok with `x-expires` parameters — typically a few hours. Download or pin them immediately if needed. - **Region-locked content.** Some hashtags return different videos depending on proxy IP region. Set `countryCode` to pin reproducible results. - **`commentsPerPost` adds wall-clock time linearly.** Each video requires its own navigation + scroll cycle (~10–30s depending on how many comments load). 50 videos × 50 comments easily adds 10+ minutes to a run. Use it sparingly, or run separate scrapes for the video list and the comments if you only need comments on a small subset. - **Only top-level comments.** Reply threads under each comment are not fetched. `replyCount` tells you how many replies exist if you need to spot threads worth fetching by other means. ### Contact Found a bug or want a feature? Email . For more scrapers, browse the [Apify Store](https://apify.com/store). # Actor input Schema ## `startUrls` (type: `array`): TikTok hashtag URLs (https://www.tiktok.com/tag/) or search URLs (https://www.tiktok.com/search?q=). Other TikTok URL types (user profiles, individual videos) are not supported. ## `maxItems` (type: `integer`): Maximum number of videos to collect per hashtag or search URL. ## `commentsPerPost` (type: `integer`): How many comments to fetch per video (embedded into each video record under `comments`). Set to 0 to skip comments entirely. Note: enabling this navigates to each video's page individually, so a large value across many videos significantly increases run time. ## `countryCode` (type: `string`): Country for the residential proxy exit node. Use this when the target site serves different content by region. ## `fuzzySearch` (type: `string`): Fuzzy search the results and print them to console. ## Actor input object example ```json { "startUrls": [ { "url": "https://www.tiktok.com/tag/funny" } ], "maxItems": 10, "commentsPerPost": 0, "countryCode": "US", "fuzzySearch": "love" } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "startUrls": [ { "url": "https://www.tiktok.com/tag/funny" } ], "maxItems": 10, "commentsPerPost": 0, "fuzzySearch": "love" }; // Run the Actor and wait for it to finish const run = await client.actor("dtrungtin/tiktok-hashtag-search-profile-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "startUrls": [{ "url": "https://www.tiktok.com/tag/funny" }], "maxItems": 10, "commentsPerPost": 0, "fuzzySearch": "love", } # Run the Actor and wait for it to finish run = client.actor("dtrungtin/tiktok-hashtag-search-profile-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrls": [ { "url": "https://www.tiktok.com/tag/funny" } ], "maxItems": 10, "commentsPerPost": 0, "fuzzySearch": "love" }' | apify call dtrungtin/tiktok-hashtag-search-profile-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=dtrungtin/tiktok-hashtag-search-profile-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "TikTok Hashtag, Search & Profile Scraper", "description": "Extract TikTok videos from any hashtag page without dealing with TikTok's anti-bot defenses, sign-in walls, or undocumented APIs.", "version": "0.0", "x-build-id": "sukbwJkVaKzhDAENx" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/dtrungtin~tiktok-hashtag-search-profile-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-dtrungtin-tiktok-hashtag-search-profile-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/dtrungtin~tiktok-hashtag-search-profile-scraper/runs": { "post": { "operationId": "runs-sync-dtrungtin-tiktok-hashtag-search-profile-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/dtrungtin~tiktok-hashtag-search-profile-scraper/run-sync": { "post": { "operationId": "run-sync-dtrungtin-tiktok-hashtag-search-profile-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "startUrls", "maxItems" ], "properties": { "startUrls": { "title": "Hashtag or search URLs", "type": "array", "description": "TikTok hashtag URLs (https://www.tiktok.com/tag/) or search URLs (https://www.tiktok.com/search?q=). Other TikTok URL types (user profiles, individual videos) are not supported.", "items": { "type": "object", "required": [ "url" ], "properties": { "url": { "type": "string", "title": "URL of a web page", "format": "uri" } } } }, "maxItems": { "title": "Max items per URL", "minimum": 1, "maximum": 500, "type": "integer", "description": "Maximum number of videos to collect per hashtag or search URL.", "default": 10 }, "commentsPerPost": { "title": "Comments per video", "minimum": 0, "maximum": 20, "type": "integer", "description": "How many comments to fetch per video (embedded into each video record under `comments`). Set to 0 to skip comments entirely. Note: enabling this navigates to each video's page individually, so a large value across many videos significantly increases run time.", "default": 0 }, "countryCode": { "title": "Proxy Country", "enum": [ "US", "DE", "VN", "FR", "GB" ], "type": "string", "description": "Country for the residential proxy exit node. Use this when the target site serves different content by region.", "default": "US" }, "fuzzySearch": { "title": "Fuzzy search", "type": "string", "description": "Fuzzy search the results and print them to console." } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```