Baidu Tieba Public Threads Scraper

Search public Baidu Tieba forums or collect known public thread URLs. The Actor discovers forum listings with explicit pagination, optionally enriches selected threads, and returns normalized thread records with search provenance.

Input

Use query or queries for forum searches, threadUrls for direct public threads, or combine both. The full accepted contract is in .actor/input_schema.json.

Field	Type	Default	Description
query	string	—	Forum name to search, such as 人工智能.
queries	array of strings	—	Batch of forum names, selected fairly across forums.
threadUrls	array	—	Public https://tieba.baidu.com/p/<id> URLs.
maxItems	integer	20	Global maximum number of thread records.
maxItemsPerQuery	integer	20	Maximum candidates selected from each forum before fair merging.
maxPages	integer	3	Maximum forum listing pages per forum.
maxConcurrency	integer	4	Maximum listing/thread pages processed concurrently.
maxPostsPerThread	integer	50	Maximum post objects retained from a rendered thread page.
minReplies	integer	0	Minimum published reply count for a listing candidate.
sortBy	string	searchOrder	searchOrder, replyCountDesc, lastReplyDesc, or titleAsc.
includeKeywords	array of strings	[]	Keep a thread only when every keyword appears in its title, excerpt, author, or forum name.
excludeKeywords	array of strings	[]	Skip a thread when any keyword appears in its title, excerpt, author, or forum name.
includeThreadDetails	boolean	false	Open each public thread for posts and richer metadata. Disabled mode is faster and less likely to encounter thread restrictions.
includePosts	boolean	true	Include normalized posts loaded on each opened thread page.
includeImages	boolean	true	Include public image URLs from loaded posts.
includeVideos	boolean	true	Include public video or poster URLs from loaded posts.
includeSticky	boolean	true	Include threads marked sticky in forum listings.
featuredOnly	boolean	false	Keep only threads marked featured or 精品.
failOnNoResults	boolean	false	Fail instead of returning an empty dataset when no public threads are available.
maxRequestRetries	integer	2	Bounded retries for temporary network, proxy, rate-limit, or browser failures.
navigationTimeoutSecs	integer	45	Maximum time per browser navigation.
requestHandlerTimeoutSecs	integer	90	Maximum time per listing or thread handler.
requestDelayMs	integer	200	Delay before each listing/thread navigation.
waitAfterLoadMillis	integer	750	Bounded wait for public DOM content immediately after navigation.
blockMedia	boolean	true	Abort media and font requests after confirming they are not needed for requested fields.
proxyConfiguration	object	—	Optional authorized Apify Residential or custom HTTP proxy settings. GOOGLE_SERP is incompatible.

Example:

{
  "query": "人工智能",
  "maxItems": 20,
  "maxPages": 2,
  "includeThreadDetails": false
}

Output

Records follow .actor/dataset_schema.json. A record identifies a thread and preserves canonical URL, source, extraction method, and scrape time. Optional post, author, media, and timestamp values are omitted when Tieba does not provide them.

When includeThreadDetails is enabled and Baidu refuses an individual thread page with a verification challenge, the verified forum listing is kept as recordType: "forum-listing" with detailStatus: "unavailable" and a detailFailureCode (for example TARGET_VERIFICATION). Only fields visible on the listing card are included, and the run reports status: "partial" with threadPagesFailed, listingFallbackRecords, and threadFailureCodes in OUTPUT. Nothing is fabricated for the missing thread page.

How to use

Enter a forum search or public thread URL in the Apify Input tab. Leave includeThreadDetails disabled for the faster listing mode, or enable it when you need opening posts and loaded replies. Review results in the Output dataset and run counts in the OUTPUT key-value record.

Proxy and access behavior

Tieba may return HTTP 403 or an interactive verification page. The Actor detects these responses and fails closed; it never stores a block page and does not solve or bypass CAPTCHAs. Authorized Apify Residential or custom HTTP proxies may be supplied in proxyConfiguration; GOOGLE_SERP is unsupported. Local Apify Proxy use may require the account's Proxy external-access feature. When local DNS cannot resolve a Baidu host, the Actor can use a bounded public DNS lookup to configure the official hostname without changing the requested host.

replyCount is Tieba's published total. loadedPostCount counts post objects visible on the rendered page and is not a substitute for the published total.