# Baidu Search Scraper (`crawlerbros/baidu-search-scraper`) Actor Scrape Baidu search results - web, image, and news. Extract titles, URLs, snippets, dates, and image metadata for any query with advanced filters (time range, filetype, language, site restriction, exact phrase, exclude words). - **URL**: https://apify.com/crawlerbros/baidu-search-scraper.md - **Developed by:** [Crawler Bros](https://apify.com/crawlerbros) (community) - **Categories:** Automation, Developer tools, SEO tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $3.00 / 1,000 results This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js/docs.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python/docs.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Baidu Search Scraper Extract search results from **Baidu** — China's largest search engine with 70%+ market share. Supports web search, image search, and news search with powerful filtering options including time range, file type, language, site restriction, and more. ### What you can scrape - **Web results** — organic search listings with titles, URLs, snippets, and dates - **Image results** — full image URLs, thumbnails, dimensions, and source pages - **News results** — news articles with publisher, date, and excerpt ### Use cases - **SEO & market research** — analyze how Chinese-language content ranks on Baidu - **Competitor intelligence** — monitor brand mentions and competitor content across Chinese web - **Academic research** — gather Chinese-language sources and academic content - **Media monitoring** — track news coverage and sentiment on any topic - **Content gap analysis** — discover popular content and trending queries in China - **Lead generation** — find businesses, directories, and profiles related to your niche ### Input | Field | Type | Description | |---|---|---| | `queries` | array | One or more search queries (required) | | `mode` | select | `webSearch`, `imageSearch`, or `newsSearch` | | `maxResults` | integer | Maximum results per query (1–500, default 50) | | `timeRange` | select | `any`, `day`, `week`, `month`, `year`. Applies to `webSearch` and `newsSearch` only — Baidu image search has no time filter. | | `language` | select | `any`, `simplified` (Simplified Chinese), `traditional` (Traditional Chinese). `webSearch` only. | | `filetype` | select | `any`, `pdf`, `doc`, `xls`, `ppt`, `rtf`. `webSearch` only. | | `siteRestrict` | string | Restrict results to a domain (e.g. `wikipedia.org`). `webSearch` only. | | `exactPhrase` | string | Results must contain this exact phrase. `webSearch` only. | | `excludeWords` | array | Words to exclude from results. `webSearch` only. | | `titleOnly` | boolean | Match search terms in page titles only. `webSearch` only. | | `resolveUrls` | boolean | Follow Baidu's redirect links to extract the true destination URL for results where it wasn't already extracted from the page. `webSearch` only. Default `true`. | | `proxyConfiguration` | object | Apify Proxy settings (recommended for non-Chinese IPs) | #### Example input (web search) ```json { "mode": "webSearch", "queries": ["人工智能 最新进展", "artificial intelligence China"], "maxResults": 50, "timeRange": "month", "language": "simplified", "proxyConfiguration": { "useApifyProxy": true } } ```` #### Example input (image search) ```json { "mode": "imageSearch", "queries": ["Great Wall of China"], "maxResults": 100, "proxyConfiguration": { "useApifyProxy": true } } ``` #### Example input (news search) ```json { "mode": "newsSearch", "queries": ["technology innovation"], "maxResults": 30, "timeRange": "week", "proxyConfiguration": { "useApifyProxy": true } } ``` ### Output #### Web search result ```json { "query": "artificial intelligence", "position": 1, "page": 1, "resultType": "organic", "title": "What is Artificial Intelligence? - Wikipedia", "url": "https://en.wikipedia.org/wiki/Artificial_intelligence", "baiduUrl": "https://www.baidu.com/link?url=abc123", "displayUrl": "en.wikipedia.org", "snippet": "Artificial intelligence (AI) is intelligence demonstrated by machines...", "date": "2024-01-15", "scrapedAt": "2024-06-01T10:30:00+00:00" } ``` #### Image search result ```json { "query": "Great Wall of China", "position": 1, "title": "Great Wall of China at sunset", "imageUrl": "https://images.example.com/greatwall_full.jpg", "thumbnailUrl": "https://images.example.com/greatwall_thumb.jpg", "sourceUrl": "https://travel.example.com/china-great-wall", "sourceDomain": "travel.example.com", "width": 1920, "height": 1080, "scrapedAt": "2024-06-01T10:30:00+00:00" } ``` #### News search result ```json { "query": "technology innovation", "position": 1, "page": 1, "title": "China tech firms unveil AI breakthroughs at annual summit", "url": "https://news.example.com/china-ai-summit", "source": "Reuters", "date": "2024-05-28", "snippet": "Leading Chinese technology companies showcased their latest...", "scrapedAt": "2024-06-01T10:30:00+00:00" } ``` #### Related search result (web mode) ```json { "query": "artificial intelligence", "position": 11, "page": 1, "resultType": "related_search", "searchTerm": "machine learning applications", "scrapedAt": "2024-06-01T10:30:00+00:00" } ``` ### Proxy recommendation Baidu is a Chinese search engine. From non-Chinese IP addresses (most cloud/datacenter IPs outside China), Baidu may throttle or redirect requests. **Using Apify Proxy with the AUTO datacenter group is strongly recommended** to get reliable results from a broad pool of IPs. Enable proxy in the input: ```json { "proxyConfiguration": { "useApifyProxy": true } } ``` ### Search operators (web mode) The scraper transparently applies Baidu search operators from your input fields: | Input field | Operator applied | Example | |---|---|---| | `siteRestrict` | `site:domain.com` | `site:wikipedia.org` | | `exactPhrase` | `"exact phrase"` | `"machine learning"` | | `excludeWords` | `-word` for each | `-java -ruby` | | `titleOnly` | `intitle:query` | `intitle:artificial intelligence` | ### Limitations - Baidu personalises results for Chinese IP addresses; non-Chinese IPs may see different result sets - Very rapid scraping (hundreds of requests per minute) may trigger CAPTCHA — use proxy and keep `maxResults` reasonable - Baidu image search returns results from Baidu's own index, which may differ from Google Images for the same query - News results reflect Baidu News sources; international English-language news may have limited coverage ### FAQ **Why am I getting 0 results?** Enable `proxyConfiguration` with `useApifyProxy: true`. Baidu often blocks datacenter IP addresses from outside China. **What languages does Baidu search support?** Baidu indexes content in all languages but is optimised for Chinese. The `language` filter restricts results to Simplified or Traditional Chinese script. **Can I scrape more than 500 results?** The current maximum is 500 results per query. For larger datasets, run multiple queries with different keywords. **What is a "related search" result?** Baidu displays a "Related Searches" box on many result pages. These are captured as records with `resultType: "related_search"` and a `searchTerm` field. **Does the scraper resolve Baidu redirect URLs?** Yes. In most cases the direct destination URL is already extracted from Baidu's `mu` attribute — no additional HTTP request needed. In the rare fallback case where only a Baidu redirect link (`baiduUrl`) is available, the scraper follows it and replaces `url` with the resolved destination when `resolveUrls` is enabled (default `true`). `baiduUrl` is always kept alongside `url` when present, for traceability. # Actor input Schema ## `mode` (type: `string`): Which Baidu vertical to scrape. ## `queries` (type: `array`): One or more search terms. Each query is scraped independently up to maxResults. ## `maxResults` (type: `integer`): Maximum number of results to return for each query (across all pages). ## `timeRange` (type: `string`): Restrict results to a publication time window. Applies to webSearch and newsSearch modes only; Baidu image search has no time filter. ## `language` (type: `string`): Restrict results to pages in a specific Chinese script. Only applies to webSearch mode. ## `filetype` (type: `string`): Restrict results to a specific document format. Only applies to webSearch mode. ## `siteRestrict` (type: `string`): Limit results to a specific website domain (e.g. wikipedia.org). Equivalent to the site: operator. Only applies to webSearch mode. ## `exactPhrase` (type: `string`): Require results containing this exact phrase. Wrapped in quotes automatically. Only applies to webSearch mode. ## `excludeWords` (type: `array`): Words to exclude from results. Each is prefixed with the minus (-) operator. Only applies to webSearch mode. ## `titleOnly` (type: `boolean`): When enabled, Baidu matches search terms in page titles only (intitle: operator). Only applies to webSearch mode. ## `resolveUrls` (type: `boolean`): When enabled, follows Baidu redirect links to extract the actual destination URL. The real URL is always extracted from the page when available, so this mainly affects fallback behaviour. Only applies to webSearch mode. ## `proxyConfiguration` (type: `object`): Proxy settings for accessing Baidu. Using Apify Proxy (AUTO datacenter group) is strongly recommended for non-Chinese IP addresses to avoid blocks. ## Actor input object example ```json { "mode": "webSearch", "queries": [ "artificial intelligence" ], "maxResults": 50, "timeRange": "any", "language": "any", "filetype": "any", "excludeWords": [], "titleOnly": false, "resolveUrls": true, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [] } } ``` # Actor output Schema ## `results` (type: `string`): Dataset containing all scraped Baidu search results. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "mode": "webSearch", "queries": [ "artificial intelligence" ], "maxResults": 50, "timeRange": "any", "language": "any", "filetype": "any", "excludeWords": [], "titleOnly": false, "resolveUrls": true, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [] } }; // Run the Actor and wait for it to finish const run = await client.actor("crawlerbros/baidu-search-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "mode": "webSearch", "queries": ["artificial intelligence"], "maxResults": 50, "timeRange": "any", "language": "any", "filetype": "any", "excludeWords": [], "titleOnly": False, "resolveUrls": True, "proxyConfiguration": { "useApifyProxy": True, "apifyProxyGroups": [], }, } # Run the Actor and wait for it to finish run = client.actor("crawlerbros/baidu-search-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "mode": "webSearch", "queries": [ "artificial intelligence" ], "maxResults": 50, "timeRange": "any", "language": "any", "filetype": "any", "excludeWords": [], "titleOnly": false, "resolveUrls": true, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [] } }' | apify call crawlerbros/baidu-search-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=crawlerbros/baidu-search-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Baidu Search Scraper", "description": "Scrape Baidu search results - web, image, and news. Extract titles, URLs, snippets, dates, and image metadata for any query with advanced filters (time range, filetype, language, site restriction, exact phrase, exclude words).", "version": "1.0", "x-build-id": "LRuWQF9X9KmxGrhyH" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/crawlerbros~baidu-search-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-crawlerbros-baidu-search-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/crawlerbros~baidu-search-scraper/runs": { "post": { "operationId": "runs-sync-crawlerbros-baidu-search-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/crawlerbros~baidu-search-scraper/run-sync": { "post": { "operationId": "run-sync-crawlerbros-baidu-search-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "queries" ], "properties": { "mode": { "title": "Search mode", "enum": [ "webSearch", "imageSearch", "newsSearch" ], "type": "string", "description": "Which Baidu vertical to scrape.", "default": "webSearch" }, "queries": { "title": "Search queries", "type": "array", "description": "One or more search terms. Each query is scraped independently up to maxResults.", "default": [ "artificial intelligence" ], "items": { "type": "string" } }, "maxResults": { "title": "Max results per query", "minimum": 1, "maximum": 500, "type": "integer", "description": "Maximum number of results to return for each query (across all pages).", "default": 50 }, "timeRange": { "title": "Time range", "enum": [ "any", "day", "week", "month", "year" ], "type": "string", "description": "Restrict results to a publication time window. Applies to webSearch and newsSearch modes only; Baidu image search has no time filter.", "default": "any" }, "language": { "title": "Language filter", "enum": [ "any", "simplified", "traditional" ], "type": "string", "description": "Restrict results to pages in a specific Chinese script. Only applies to webSearch mode.", "default": "any" }, "filetype": { "title": "File type filter", "enum": [ "any", "pdf", "doc", "xls", "ppt", "rtf" ], "type": "string", "description": "Restrict results to a specific document format. Only applies to webSearch mode.", "default": "any" }, "siteRestrict": { "title": "Restrict to domain", "type": "string", "description": "Limit results to a specific website domain (e.g. wikipedia.org). Equivalent to the site: operator. Only applies to webSearch mode." }, "exactPhrase": { "title": "Exact phrase", "type": "string", "description": "Require results containing this exact phrase. Wrapped in quotes automatically. Only applies to webSearch mode." }, "excludeWords": { "title": "Exclude words", "type": "array", "description": "Words to exclude from results. Each is prefixed with the minus (-) operator. Only applies to webSearch mode.", "default": [], "items": { "type": "string" } }, "titleOnly": { "title": "Search title only", "type": "boolean", "description": "When enabled, Baidu matches search terms in page titles only (intitle: operator). Only applies to webSearch mode.", "default": false }, "resolveUrls": { "title": "Resolve Baidu redirect URLs", "type": "boolean", "description": "When enabled, follows Baidu redirect links to extract the actual destination URL. The real URL is always extracted from the page when available, so this mainly affects fallback behaviour. Only applies to webSearch mode.", "default": true }, "proxyConfiguration": { "title": "Proxy configuration", "type": "object", "description": "Proxy settings for accessing Baidu. Using Apify Proxy (AUTO datacenter group) is strongly recommended for non-Chinese IP addresses to avoid blocks." } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```