Baidu Videos Scraper

Search Baidu's public video results and export normalized, API-ready video records. The Actor uses a persistent Playwright browser, validates each response, accepts only verified Baidu video templates, resolves safe Baidu redirects to canonical destinations, and writes a stable dataset plus an OUTPUT run summary.

Input modes

• Single query: set query.
• Batch search: set queries with up to 20 phrases. Duplicate queries are removed and output is interleaved fairly.
• Single item: set maxItems and maxItemsPerQuery to 1.
• Multiple offsets: raise maxPages. The Actor stops when an offset contains no new verified video templates or no accepted records.
• No results: a valid search with explicit no-result text completes with an empty dataset and OUTPUT.status = "no-results".

You never need to construct or paste a Baidu result-page URL.

Input

Section	Fields	Purpose
Search	query, queries, market	Search phrase(s) and browser locale
Limits	maxItems, maxItemsPerQuery, maxPages	Total, per-query, and offset limits
Filters and sorting	includeKeywords, excludeKeywords, includePlatforms, excludePlatforms, minViewCount, maxDurationSeconds, includeShorts, sortBy	Narrow and order normalized records
Developer options	maxConcurrency, maxRequestRetries, navigationTimeoutSecs, requestHandlerTimeoutSecs, requestDelayMs, resolveRedirects, blockMedia	Tune bounded concurrency, retries, timeouts, pacing, redirect resolution, and resources
Proxy	proxyConfiguration	Direct access, an authorized Apify Proxy, or custom proxy URLs

Example batch input:

{
  "queries": ["机器学习 教程", "人工智能 教程"],
  "maxItems": 20,
  "maxItemsPerQuery": 10,
  "maxPages": 2,
  "maxConcurrency": 2,
  "includeKeywords": ["教程"],
  "sortBy": "viewsDesc",
  "requestDelayMs": 500,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"],
    "countryCode": "CN"
  }
}

GOOGLE_SERP cannot be used with Baidu. It is an HTTP proxy specialized for supported Google Search and Google Shopping traffic. For Baidu, use direct access, an authorized Residential proxy, or custom proxy URLs.

Output

The dataset schema documents the rich normalized record contract, including:

• Stable identity and ordering: id, global/query/result/page positions.
• Text: title, sanitized highlighted title, highlighted terms, description, and snippet when displayed.
• Destinations: canonical watch/source URLs, domain, platform, and video ID.
• Media: thumbnail and preview image URLs.
• Video metadata: duration, parsed duration seconds, displayed and parsed view counts, likes, comments, publication date, short-video and advertisement flags.
• Provenance: extraction method, engine metadata, query, locale, search URL, and ISO scrape timestamp.

Optional fields that Baidu does not expose are omitted recursively. The Actor never stores challenge pages, ordinary non-video cards, unresolved Baidu redirects, unsafe HTML, secrets, or fabricated placeholder records.

The OUTPUT key-value record contains safe run metrics: status, requested queries, limits, developer settings, filters, pages fetched/failed, verified cards, filtered count, per-query metrics, connection mode, DNS mode, failure categories, and completion time. It also records the last HTTP status, response content type, final page host, and sanitized error code per query. It does not contain cookies, authorization headers, proxy credentials, or raw response bodies.

Pagination and batch behavior

Each query is crawled independently across Baidu's pn offsets. Results are deduplicated by canonical URL and stable ID, capped per query, then interleaved fairly before the global limit and final sort are applied. A later failed query cannot silently turn into a successful complete run: the summary records failed query positions and the final status is partial only when usable records were produced.

Proxy and access behavior

The Actor first tries ordinary browser navigation. When the host cannot be resolved locally, it uses bounded public DNS-over-HTTPS lookups after checking HTTP status and JSON content type, then supplies validated host mappings to Chrome. This is a transport fallback, not an access-control bypass.

An authorized Apify Proxy or custom proxy can be supplied through proxyConfiguration. Cookies are kept inside the current browser context, response status and HTML content type are validated before extraction, and temporary transport/rate-limit/time-out errors receive bounded retries. Baidu verification, CAPTCHA, and access-denied pages are detected, marked as target verification, and fail closed without solving or bypassing them.

Development

npm ci --ignore-scripts
npm test
npm run check
apify validate-schema
npm run validate:dataset
npm run validate:readme
apify run --purge --input-file INPUT.json
npm run validate:run

npm run validate:run inspects the current local Actor storage directly. Dataset records are stored under storage/datasets/default, and the safe run summary is stored at storage/key_value_stores/default/OUTPUT.json.

Current source limitations

Baidu may expose a video carousel on the first offset and ordinary web cards on later offsets. Ordinary cards are not videos and are never stored, so a run can return fewer items than requested. Some layouts do not display descriptions, view counts, dates, or thumbnails; those fields remain absent rather than inferred.

Repeated direct requests can trigger Baidu's interactive verification. The Actor detects this condition, records lastErrorCode: "TARGET_VERIFICATION" and the failure category in OUTPUT, and does not attempt to solve or bypass the challenge. Use an authorized China Residential proxy in Apify Cloud when direct access is rate-limited.