# Baidu News Scraper (`searchapi/baidu-news-scraper`) Actor

Scrape news search results from Baidu (China's #1 search engine). Extracts article titles, snippets, sources, authors, publication dates, and thumbnails. Supports pagination, source filters, and Chinese date parsing.

- **URL**: https://apify.com/searchapi/baidu-news-scraper.md
- **Developed by:** [Search API](https://apify.com/searchapi) (community)
- **Categories:** News, Developer tools, Automation
- **Stats:** 2 total users, 1 monthly users, 69.2% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.99 / 1,000 search results

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Baidu News Scraper

Search Baidu's public News interface and export normalized articles with stable IDs, canonical article URLs, publisher metadata, snippets, publication dates, images, query context, and extraction provenance.

### Features

- Uses the current `news.baidu.com/ns` search interface and follows Baidu's validated native pagination links.
- Supports `single` and fair `batch` query modes.
- Enforces total and per-query record limits, with bounded query concurrency.
- Resolves Baidu article redirects when enabled and removes common tracking parameters.
- Produces stable SHA-256 IDs and deduplicates by canonical article URL.
- Omits null, blank, empty-array, and empty-object fields recursively.
- Buffers the complete run before writing, so a later verification challenge cannot publish a partial dataset.
- Supports direct access, Apify Residential Proxy, and custom proxy URLs through Apify's proxy input editor.
- Uses the matched, pinned Playwright Firefox runtime with system DNS, verified TLS, and default browser security settings. No host mappings or fallback DNS resolvers are used.
- Supports request pacing, source/title/snippet include/exclude filters, stable source/newest/title sorting, and a validated `OUTPUT` run summary.
- Stops the whole run on login, CAPTCHA/verification, or HTTP 401/403/407/429. These gates are never retried and no proxy or browser identity is rotated; all buffered rows are discarded.

`GOOGLE_SERP` Proxy is not supported because it is restricted to Google Search and Google Shopping, not Baidu.

### Input

| Field | Description | Default |
| --- | --- | --- |
| `mode` | `single` or `batch` | `single` |
| `query` | Query used in single mode | `人工智能` |
| `queries` | Up to 20 deduplicated queries used in batch mode | `[]` |
| `maxItems` | Maximum records across the run | `50` |
| `maxItemsPerQuery` | Maximum records collected for each query | `50` |
| `maxPages` | Maximum pages followed for each query | `5` |
| `resultsPerPage` | Requested Baidu page size; live pagination links remain authoritative | `20` |
| `maxConcurrency` | Concurrent query browser pages | `2` |
| `requestDelayMs` | Delay before each page navigation | `200` |
| `resolveRedirects` | Resolve Baidu article redirect links | `true` |
| `market` | Locale tag used in output context | `zh-CN` |
| `maxRequestRetries` | Retries for temporary DNS, transport, timeout, or server failures | `3` |
| `navigationTimeoutSecs` | Per-navigation timeout | `60` |
| `requestHandlerTimeoutSecs` | Per-query handler timeout | `120` |
| `sortBy` | `sourceOrder`, `publishedAtDesc`, or `titleAsc` | `sourceOrder` |
| `includeSources` / `excludeSources` | Publisher substring filters | `[]` |
| `includeKeywords` / `excludeKeywords` | Title/snippet substring filters | `[]` |
| `proxyConfiguration` | Direct, Apify Proxy, or custom proxy configuration | direct |

### Output

The documented dataset schema includes these typed fields:

- Identity: `id`, `type`, `resultType`, `position`, `globalPosition`, `queryPosition`, `resultPosition`, `page`
- Article: `title`, `titleHtml`, `snippet`, `snippetHtml`, `description`, `emphasis`
- URLs: `link`, `url`, `domain`, `displayUrl`, `sourceUrl`, `sourceDomain`, `searchUrl`
- Publisher and dates: `source`, `publishedAt`, `publishedAtRaw`, `relativeTime`, `date`
- Media: `thumbnail`, `thumbnailUrl`, `imageUrl`, `images`, dimensions, source logo/favicon
- Context: `query`, `searchQuery`, `market`, `country`, `language`, `searchMetadata`
- Provenance: `engineId`, `engineData`, `extractionMethod`, `snippetExtractionMethod`, `scrapedAt`

Optional fields are omitted when Baidu does not expose them. No placeholder or fabricated values are added.

`publishedAtRaw`, `relativeTime`, and `date` preserve displayed date labels. Relative labels (such as `6小时前` or `3天前`) and dates missing a year do not supply an exact publication instant, so `publishedAt` is omitted. A complete calendar date stays `YYYY-MM-DD`; an explicit ISO timestamp with a timezone retains its supplied precision. No extraction-time-based estimate is emitted. Newest-first sorting can use only the absolute dates the source actually supplies.

Article `thumbnail`, `thumbnailUrl`, `imageUrl`, dimensions, and `images` exclude publisher logos and author avatars. Publisher-only cards keep `sourceLogoUrl`/`favicon` and omit article-image fields.

### Examples

Single query:

```json
{
  "mode": "single",
  "query": "人工智能",
  "maxItems": 20,
  "maxPages": 2,
  "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"], "countryCode": "CN" }
}
```

Fair batch:

```json
{
  "mode": "batch",
  "queries": ["人工智能", "机器人"],
  "maxItems": 20,
  "maxItemsPerQuery": 10,
  "maxConcurrency": 2,
  "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"], "countryCode": "CN" }
}
```

### Offline readiness and first run

The Console prefill and checked-in `INPUT.json` request five records from one page, directly, with one worker and no retries or article-redirect requests. Runtime defaults in the table above are unchanged when those inputs are omitted.

Run `npm ci`, then `npm run check` for syntax checks, deterministic DOM/handler/runtime/schema regressions, and a real SDK startup with local storage and network calls disabled. Synthetic test fixtures are never live source evidence.

The build uses `apify/actor-node-playwright-firefox:22-1.61.1` with Playwright `1.61.1` and checks that Firefox is installed. No custom browser preferences or sandbox/TLS exceptions are configured.

`OUTPUT.json` records last HTTP status, response content type, final page host, and sanitized error code. It does not store response bodies or verification URLs. Article redirects are inspected one hop at a time and stop at denial or a verification/login location; publisher Location URLs can be returned without fetching the publisher page.

This readiness revision requires a fresh cloud build and live QA before current extraction success can be claimed. If Baidu gates that run, it must fail with an empty dataset and be reported as blocked.

# Changelog

This Actor's version history is a separate document: https://apify.com/searchapi/baidu-news-scraper/changelog.md

# Actor input Schema

## `mode` (type: `string`):

Use one query or a fair batch of multiple queries.

## `query` (type: `string`):

News search query (Chinese queries recommended)

## `queries` (type: `array`):

Queries used in batch mode. Duplicate and blank values are removed.

## `maxItems` (type: `integer`):

Maximum number of articles to scrape

## `maxPages` (type: `integer`):

Maximum number of pagination pages to follow

## `maxItemsPerQuery` (type: `integer`):

Per-query cap before the fair global maxItems limit is applied.

## `resultsPerPage` (type: `integer`):

Number of results requested from Baidu on each page.

## `resolveRedirects` (type: `boolean`):

Follow Baidu /link?url=... redirects to get the real article URL

## `market` (type: `string`):

Locale tag for the search (e.g. zh-CN)

## `maxRequestRetries` (type: `integer`):

Retry temporary DNS, browser, network, and server failures. Verification, login, denied access, and rate limits terminate the entire run without retries.

## `maxConcurrency` (type: `integer`):

Number of query browser pages processed concurrently.

## `requestDelayMs` (type: `integer`):

Delay before each Baidu News page navigation.

## `navigationTimeoutSecs` (type: `integer`):

Maximum time allowed for each page navigation.

## `requestHandlerTimeoutSecs` (type: `integer`):

Maximum time allowed for extraction and redirect resolution.

## `sortBy` (type: `string`):

Ordering applied after fair multi-query interleaving and deduplication.

## `includeSources` (type: `array`):

Keep records whose publisher contains at least one value.

## `excludeSources` (type: `array`):

Remove records whose publisher contains any value.

## `includeKeywords` (type: `array`):

Keep records whose title or snippet contains at least one value.

## `excludeKeywords` (type: `array`):

Remove records whose title or snippet contains any value.

## `proxyConfiguration` (type: `object`):

Proxy configuration for the scraper

## Actor input object example

```json
{
  "mode": "single",
  "query": "人工智能",
  "queries": [],
  "maxItems": 5,
  "maxPages": 1,
  "maxItemsPerQuery": 5,
  "resultsPerPage": 20,
  "resolveRedirects": true,
  "market": "zh-CN",
  "maxRequestRetries": 3,
  "maxConcurrency": 1,
  "requestDelayMs": 200,
  "navigationTimeoutSecs": 60,
  "requestHandlerTimeoutSecs": 120,
  "sortBy": "sourceOrder",
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "query": "人工智能",
    "maxItems": 5,
    "maxPages": 1,
    "maxItemsPerQuery": 5,
    "resolveRedirects": false,
    "market": "zh-CN",
    "maxRequestRetries": 0,
    "maxConcurrency": 1,
    "proxyConfiguration": {
        "useApifyProxy": false
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("searchapi/baidu-news-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "query": "人工智能",
    "maxItems": 5,
    "maxPages": 1,
    "maxItemsPerQuery": 5,
    "resolveRedirects": False,
    "market": "zh-CN",
    "maxRequestRetries": 0,
    "maxConcurrency": 1,
    "proxyConfiguration": { "useApifyProxy": False },
}

# Run the Actor and wait for it to finish
run = client.actor("searchapi/baidu-news-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "query": "人工智能",
  "maxItems": 5,
  "maxPages": 1,
  "maxItemsPerQuery": 5,
  "resolveRedirects": false,
  "market": "zh-CN",
  "maxRequestRetries": 0,
  "maxConcurrency": 1,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}' |
apify call searchapi/baidu-news-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,searchapi/baidu-news-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/mEzx80rKuppyoDadz/builds/lm4jlwhkLRsxAe3pH/openapi.json