Baidu Search Scraper By Site & File Type

Baidu Search Scraper — Site-Restricted Results, File Types and Own-Domain Flags

This Baidu search scraper returns organic listings, answer boxes, related videos, people-also-search-for, related searches, and top searches for any query — narrowed to specific domains with sites, a document type with fileType, an exact phrase, or excluded terms. Every row carries the exact wd string sent to Baidu (appliedQuery), the resolved outbound link (realLink), and an own-domain match flag (matchesTargetDomain). Built for SEO auditors, brand-monitoring teams, and market researchers who need Baidu coverage on named sites rather than the open web. Results stream to the Apify dataset row by row as each query resolves.

What is the Baidu Search Scraper By Site & File Type?

It's an Apify Actor that submits search terms (or Baidu search URLs) to Baidu, composes each into a filtered query using Baidu's site:, filetype:, intitle:, and -word operators, and parses the resulting HTML into structured rows. No Baidu account or login is required — it scrapes the public search results page directly. Key capabilities:

• Query composition, not raw search — sites[], fileType, exactPhrase, excludeWords[], and titleOnly are layered onto every term before it's sent, and the composed string is returned on each row as appliedQuery
• Six result types per query — organic listings, answer box/knowledge panels, related videos, people-also-search-for, related searches, and top searches, distinguished by the resultType field
• Redirect resolution — Baidu wraps outbound links behind an opaque baidu.com/link?url=... redirect; this Actor follows it one hop and returns the true destination as realLink
• Own-domain flagging — set targetDomain and every link-bearing row gets matchesTargetDomain: true/false so you can isolate which results already point back to your site
• Desktop, mobile, and tablet SERPs — deviceType switches between www.baidu.com and m.baidu.com, which render different result layouts
• Automatic proxy fallback — starts with no proxy, escalates to Apify datacenter proxy, then residential (3 retries) only if Baidu blocks the request
• JSON output — pushed live to the Apify dataset, exportable to CSV, Excel, XML, or JSON from the Apify Console

What data can I extract with the Baidu Search Scraper?

Every dataset row carries a resultType (organic, answer_box, related_video, people_also_search_for, related_search, or top_search); which of the fields below are populated depends on that type.

Field	Example Value	Use Case
query	"python tutorial"	The original search term you submitted
appliedQuery	"python tutorial site:csdn.net"	The exact wd string sent to Baidu after site/filetype/phrase/exclude filters were applied
resultType	"organic"	Row category — filter or group by this
title	"python的安装(详细教程)_安装python-CSDN博客"	Result headline
link	"https://www.baidu.com/link?url=..."	Baidu's raw (often redirect-wrapped) link as printed on the SERP
realLink	"https://blog.csdn.net/xhmico/article/details/162863017"	The resolved true destination, one redirect hop unwrapped
fileType	"pdf" or null	Document extension detected on the link, or the active fileType filter as fallback
matchesTargetDomain	true / false / null	Whether the resolved link belongs to your targetDomain; null if that input was left empty
snippet	"Learn Python from scratch..."	Organic result description text
displayedLink	"www.baidu.com"	Host/path Baidu displays under the title
thumbnail	"https://..." or null	Image thumbnail attached to the result, if present
position	1	1-indexed rank within the combined pages fetched for this query
content	"..."	Answer-box body text (answer_box rows only)
source	"百度百科"	Answer-box attribution (answer_box rows only)
searchTerm	"python for beginners"	Suggested query text on related_search / people_also_search_for / top_search rows
richSnippet	"..." or null	Extra highlighted text Baidu attaches to some organic results
sitelinks	[{"title": "...", "link": "..."}]	Sub-links Baidu shows under a strong brand match (organic rows only)

Filter transparency and resolved links

appliedQuery and realLink are what make the scoped rows auditable. appliedQuery shows exactly which site:, filetype:, quoted-phrase, -exclude, and intitle: clauses were actually combined into the request — useful when a query returns zero rows and you need to see whether the site restriction excluded everything. realLink matters because Baidu serves most outbound organic links behind an opaque redirect token that isn't a usable destination on its own; this Actor resolves it live (capped at 25 resolutions per query) and falls back to the original link, never a guessed URL, if the hop fails.

Targeting and scoping fields

resultType, fileType, matchesTargetDomain, displayedLink, and position are what you filter and segment on downstream. resultType separates organic listings from answer boxes and discovery rows (related/top searches, people-also-search-for) so you can process each differently. fileType and matchesTargetDomain let you isolate "PDFs on this domain" or "everything that isn't already mine" without re-parsing links yourself.

Why not build this yourself?

Baidu does not publish a general-purpose search API for arbitrary web queries at scale — there's no equivalent of Google's Custom Search JSON API to call for organic results, answer boxes, and related searches in one response. Building this in-house means writing an HTML parser for Baidu's SERP markup (which changes without notice), handling the /link?url=... redirect scheme to get real outbound URLs, and managing IP blocking — Baidu returns 403/429/503 or a short anti-bot interstitial once request volume looks automated. This Actor handles all three: the parser, redirect resolution, and a no-proxy → datacenter-proxy → residential-proxy (3 retries) fallback ladder that only escalates when a block is actually detected.

How to use data extracted from Baidu

SEO auditors and content teams

Set sites to a competitor's domain or a list of publisher sites, leave fileType at any, and run your target keyword set. Every row's appliedQuery confirms the site: restriction actually applied, and position shows where each page ranks within the pages fetched. Pull title, realLink, and snippet into a spreadsheet to build a coverage report per domain per keyword.

Agencies auditing client sites

Set targetDomain to the client's domain and run a recurring keyword list. matchesTargetDomain: true rows are the client's own pages already ranking; false rows are everything else showing up ahead of or alongside them — the basis for a competitive-visibility report you can repeat on a schedule without re-writing the query logic each time.

Document and file-type research

Set fileType to pdf, doc, xls, ppt, or rtf to find downloadable material on a topic — spec sheets, reports, whitepapers. Combine with sites to restrict the search to known publisher or government domains, and read fileType on each row to confirm the detected extension matches what you asked for.

AI agents and automated pipelines

Because input and output are both plain JSON, an agent can call this Actor as a tool: pass a query plus sites/fileType/targetDomain, get back typed rows with resultType and realLink already resolved, and feed title+snippet into a RAG index or a downstream classification step without any HTML parsing on the agent's side.

🔼 Input sample

urls is the only required field. Everything else layers a filter on top of it or controls pagination, device, and proxy behavior.

Parameter	Required	Type	Description	Example Value
urls	Yes	array	Baidu search URLs or plain search terms, one per line	["python tutorial"]
sites	No	array	Domains to OR-restrict results to; empty = whole web	["csdn.net", "zhihu.com"]
fileType	No	string (enum, default "any")	any / pdf / doc / xls / ppt / rtf	"pdf"
exactPhrase	No	string (default "")	Phrase quoted and appended to every query	"machine learning tutorial"
excludeWords	No	array	Words appended as -word to drop matching pages	["forum", "spam"]
titleOnly	No	boolean (default false)	Appends intitle:<query> when true	true
targetDomain	No	string (default "")	Your domain; flags each row with matchesTargetDomain	"docs.python.org"
deviceType	No	string (enum, default "desktop")	desktop / mobile / tablet	"mobile"
languageLocalization	No	integer (default 1, min 1, max 3)	1 = all languages, 2 = Simplified Chinese, 3 = Traditional Chinese	1
startPage	No	integer (default 1, min 1)	Page to start scraping from	1
numResults	No	integer (default 10, min 1, max 50)	Results per page; also sent as Baidu's rn= param	10
timePeriod	No	object	{startDate, endDate} or {daysAgo}; empty = no filter	{"daysAgo": 7}
maxPagination	No	integer (default 3, min 0, max 10)	Pages to scrape per query; 0 = capped at 10	3
outputFile	No	string (default "")	Key-value store key to also save a JSON summary under	"my-run"
proxyConfiguration	No	object	Apify proxy settings; default is no proxy	{"useApifyProxy": false}

{
  "urls": ["python tutorial"],
  "sites": ["csdn.net"],
  "fileType": "any",
  "exactPhrase": "",
  "excludeWords": [],
  "titleOnly": false,
  "targetDomain": "docs.python.org",
  "deviceType": "desktop",
  "numResults": 10,
  "maxPagination": 1
}

Common pitfall: sites and targetDomain do two different things. sites narrows where Baidu searches (via site:); targetDomain only labels rows that already came back — it never filters out results. If you want results scoped to your own domain only, put it in sites, not targetDomain. ⚠️ And maxPagination: 0 does not mean "unlimited" — it's internally capped at 10 pages per query, so the real ceiling per query is numResults × 10.

🔽 Output sample

One row per result, pushed to the dataset live as each query resolves — not batched at the end of the run. Every row is flat JSON with a consistent resultType-keyed shape; export to JSON, CSV, Excel, or XML from the Apify Console. If outputFile is set, a {summary, results_by_query} JSON object is additionally saved to the key-value store under that key.

{
  "query": "python tutorial",
  "resultType": "organic",
  "title": "python的安装(详细教程)_安装python-CSDN博客",
  "link": "https://www.baidu.com/link?url=abc123...",
  "snippet": "Learn how to install and configure Python step by step...",
  "displayedLink": "www.baidu.com",
  "thumbnail": null,
  "position": 1,
  "richSnippet": null,
  "appliedQuery": "python tutorial site:csdn.net",
  "realLink": "https://blog.csdn.net/xhmico/article/details/162863017",
  "fileType": null,
  "matchesTargetDomain": false,
  "sitelinks": []
}

How do you filter and target specific search results?

The two things worth understanding before you run this Actor are how sites combines multiple domains, and how the six filters stack on a single query.

sites vs fileType — sites restricts where Baidu looks (OR-combined site: clauses), while fileType restricts what kind of document comes back. Use both together to find, say, PDFs published only on government or academic domains rather than the open web.

Stacking filters — exactPhrase, excludeWords, and titleOnly all compose onto the same query string alongside sites and fileType. appliedQuery on every output row shows you the final combined string, so you can confirm the stack behaved as intended without guessing.

Volume controls — numResults (1–50 per page) and maxPagination (0–10 pages, 0 defaulting to the 10-page cap) together bound how many organic results a single query can return; startPage lets you resume from a later page instead of always starting at page 1.

Three real examples:

{ "urls": ["annual report"], "sites": ["sec.gov"], "fileType": "pdf" }

Finds PDF annual reports hosted only on sec.gov.

{ "urls": ["machine learning"], "exactPhrase": "supervised learning", "excludeWords": ["course", "udemy"], "titleOnly": true }

Requires the exact phrase, excludes course-marketing pages, and favors results with the term in the page title.

{ "urls": ["site pricing page"], "sites": ["mybrand.com"], "targetDomain": "mybrand.com", "numResults": 50, "maxPagination": 5 }

Pulls up to 250 results restricted to your own domain, with every row flagged matchesTargetDomain: true for a self-audit.

▶️ Want to try other search-engine scrapers?

Scraper	What it extracts
Google Search Results Scraper (Question & Keyword Discovery)	Google organic results plus related searches, autocomplete completions, and People-also-ask questions expanded into answers
Google Search Results (SERP) Scraper & Shopping Prices	Google SERP rows with attached Shopping offers — merchant, price, discount, delivery, rating
DuckDuckGo Scraper — Fresh News By Date & Source	DuckDuckGo News results bounded by time window and publisher, with wire-copy duplicates grouped
Google Scholar Scraper With Institution & Journal Data	Academic papers with institution research-output stats and journal quality metrics attached

How to extract Baidu data programmatically

Run this Actor through the Apify API — one authenticated POST to start a run, then read results back from the run's dataset as structured JSON. No Baidu credentials are required; only your Apify API token.

Python example

from apify_client import ApifyClient

client = ApifyClient("<APIFY_API_TOKEN>")
run = client.actor("<YOUR_USERNAME>/baidu-search-scraper-by-site-and-file-type").call(
    run_input={
        "urls": ["python tutorial"],
        "sites": ["csdn.net"],
        "fileType": "any",
        "numResults": 10,
        "maxPagination": 2,
    }
)

for row in client.dataset(run["defaultDatasetId"]).iterate_items():
    if row.get("resultType") == "organic":
        print(row["title"], row["realLink"], row["matchesTargetDomain"])

Export to spreadsheets or CRM

From the Apify Console's dataset Export tab, download CSV or Excel directly — title, realLink, snippet, and matchesTargetDomain map cleanly onto lead-tracking or content-audit spreadsheet columns without renaming. CSV export also works through the API's dataset items endpoint with format=csv.

Is it legal to scrape Baidu?

Yes — scraping publicly accessible Baidu search results is generally legal; these are public listings that Baidu serves to any visitor without login, not gated or private content. This Actor returns search-engine result data — titles, links, snippets, and page metadata — which is business/product data rather than personal data, so GDPR's personal-data rules don't attach to it. Scraping is still bound by Baidu's own Terms of Service and applicable database-rights law in your jurisdiction, particularly around large-scale reuse or redistribution of the collected data. Consult legal counsel for commercial applications involving bulk storage of personal data.

❓ FAQ

How do I know if a result is stale or the page no longer exists?

This Actor doesn't verify link liveness after scraping — link and realLink reflect what Baidu returned at request time. Because realLink resolution is a live HTTP hop, a genuinely dead redirect target simply falls back to the original link rather than being flagged; run a separate liveness check downstream if that matters for your use case.

Can I get answer boxes and related searches along with organic results?

Yes, on the same run and same query. Every result type — organic, answer_box, related_video, people_also_search_for, related_search, top_search — is pushed as its own row, distinguished by resultType, with no separate call needed.

How accurate is the snippet and title data?

The Actor returns text exactly as Baidu's SERP renders it at request time. Accuracy depends on how current Baidu's own index is for that page; re-run the query if you need a fresher snapshot.

How many results can I get per run?

Each query returns up to numResults (max 50) results per page across up to maxPagination pages (max 10, and 0 is capped at 10 internally) — a ceiling of 500 organic results per query. Add more entries to urls to cover more queries in a single run.

How does the site restriction handle multiple domains?

List them in sites — the Actor OR-combines them into (site:a.com OR site:b.com) so a single query searches across all listed domains at once, rather than requiring one run per domain.

Does this work with Claude, ChatGPT, and AI agent frameworks?

Yes — call it as a standard HTTP endpoint via the Apify API from any agent framework capable of making authenticated REST calls; there is no Baidu-specific client library required on the agent side.

Can I use this without a Baidu account or developer key?

Yes. It scrapes Baidu's public search results directly; you only need an Apify account and API token to run the Actor itself.