Baidu Zhidao Rich Q&A Scraper

This Actor searches the public Baidu Zhidao interface and emits normalized question records with the question body, extracted answers, authors, dates, counts, categories, tags, related questions, result-card metadata, and provenance. If Baidu refuses optional question-detail pages, verified result cards are emitted as recordType: "search_result" with detailStatus: "unavailable" and a failure code; only fields visible on the card are included.

Input modes

Use the normal search form with one query or a queries array. The Actor visits the public homepage, fills the search box, follows result pagination, opens public question pages, and fairly interleaves accepted records from multiple queries.

Use questionUrls when the input is a list of public Zhidao question URLs. This switches to direct-question mode and avoids search pagination. query and queries are optional in that mode.

Example search input:

{
  "queries": ["人工智能", "机器学习"],
  "maxItems": 10,
  "maxItemsPerQuery": 5,
  "maxPages": 2,
  "sortBy": "answerCountDesc",
  "includeKeywords": ["模型"],
  "proxyConfiguration": { "useApifyProxy": false }
}

Example direct-question input:

{
  "questionUrls": [
    "https://zhidao.baidu.com/question/1832639011272142868.html"
  ],
  "maxItems": 1
}

The full input schema is in .actor/input_schema.json. Accepted fields and defaults:

Field	Type	Default
query	string	—
queries	array of strings	[]
questionUrls	array of strings	[]
market	string	zh-CN
maxItems	integer	20
maxItemsPerQuery	integer	20
maxPages	integer	3
includeKeywords	array of strings	[]
excludeKeywords	array of strings	[]
includeCategories	array of strings	[]
excludeCategories	array of strings	[]
minAnswerCount	integer	—
maxAnswerCount	integer	—
minViewCount	integer	—
resolvedOnly	boolean	false
requireBestAnswer	boolean	false
sortBy	string	searchOrder
maxConcurrency	integer	2
maxDetailConcurrency	integer	3
maxRequestRetries	integer	1
navigationTimeoutSecs	integer	45
requestHandlerTimeoutSecs	integer	180
requestDelayMs	integer	300
blockMedia	boolean	true
proxyConfiguration	object	—

Output

Dataset fields, types, and requiredness are defined in .actor/dataset_schema.json.

Each dataset item has a stable id, canonical question URLs, positional metadata, search counts, question metrics, an answers array, category breadcrumbs, related-question links, locale, extraction method, and ISO scrapedAt provenance. The OUTPUT key-value record reports pages fetched, verified cards, detail-page success/failure, filtering, pagination, query summaries, connection mode, DNS mode, and failure categories. It also provides a primary failureClass, failureCode, and sanitized failureReason when a request fails. When detail pages produce no usable record, detailFailureSamples carries up to five bounded evidence blocks (requested/final host and path, observed document title and title kind, access status, body text length, and a redacted body excerpt). Search mode preserves verified cards as partial rows; direct-question inputs and search pages without verified cards fail closed.

Search cards are emitted only after the public search page has produced verified question cards. A refused detail page does not erase those verified cards: the Actor emits a clearly marked card-only row without inventing question bodies or answers, records bounded refusal evidence in OUTPUT, and reports the run as partial. A blocked or unrecognized search page still fails closed. Empty or challenge pages are never converted into placeholder records. Duplicate question IDs and canonical URLs are removed before output.

Proxy and access behavior

Direct connections work when the target is reachable. An authorized Apify Proxy or custom proxy URL list can be supplied through proxyConfiguration. Proxy credentials are used only by the browser and are not stored in dataset records, summaries, or logs. Temporary transport, timeout, rate-limit, and DNS failures use bounded retries; interactive verification pages and target access notices are classified separately, fail closed, and are not bypassed.

The Actor uses API-safe response checks when navigation responses are available and DOM extraction for the public search and question markup. Search results are parsed before page text is checked for access markers, so matching result content does not become a false block. If Baidu changes its markup or refuses a page, the run summary reports the failure instead of fabricating data.

How to use

Choose search or direct-question mode in the Apify Input tab, enter the relevant query or public URLs, and start the Actor. Review question records in the Output dataset and the run summary in the OUTPUT key-value record.