Wikipedia Scraper — Search, Summaries & Full Text

Turn a list of names, topics or page titles into structured Wikipedia data: relevance-ranked search or exact title lookup in any of the ~340 language editions, lead summary or the complete plain-text article split into sections, categories, links, inter-language links, images, the linked Wikidata record (industry, headquarters, CEO, revenue, employees, ticker, ISIN, LEI, website, social handles…) and pageview statistics for attention tracking. Built for due-diligence and KYB teams, market researchers, content/SEO teams, and RAG pipelines.

No API key, no proxy, no browser — official Wikimedia REST and Action APIs, paced the way Wikimedia asks (one request at a time, at most ~170 a minute).

Use cases

• Company / person due diligence — includeWikidata: true gives industry, HQ, founders, CEO, parent, subsidiaries, employees, revenue (with year), ticker, ISIN, LEI, OpenCorporates & Crunchbase ids, official site and social links, plus the encyclopedic summary.
• Entity enrichment for CRM / lead lists — feed 500 company names via queries, keep fields: ["query","title","description","extract","wikidataId","url"].
• Attention / trend monitoring — includePageviews: true, pageviewsDays: 90, includeDailyPageviews: true on brands, competitors or topics; alert on spikes.
• RAG / knowledge base ingestion — mode: "full", includeSections: true gives clean plain text with a table of contents; maxTextChars caps size.
• Multilingual localisation research — includeLangLinks: true lists which languages cover a topic; run the same queries with lang: "de", "kk", "zh" for local perspectives.
• Topic mapping / SEO content clusters — includeLinks: true, includeCategories: true on seed articles to build a related-topics graph.

Input

Field	Type	Default	Allowed values / notes
queries	string[]	–	Required. One entry per lookup: free text, an entity name or an exact page title. About 1,000 summary lookups fit in the default 15-minute run, fewer with the optional blocks — see Limits.
lang	string	en	Any Wikipedia language code (en, de, fr, es, ru, zh, ja, ar, kk, uz, simple, zh-yue…). A language name (German, Deutsch) or a locale tag (en-US, pt_BR) is read as its edition, with a note in the status. See Reference.
searchMode	string	text	text (relevance search like the site search box), title (prefix match on titles), nearmatch (the page MediaWiki's "Go" box would open for the query; 0–1 result), exact (treat the query as the page title; follows redirects).
resultsPerQuery	integer	1	1–50 top hits saved per query (ignored for exact).
skipDisambiguation	boolean	false	Skip disambiguation list pages (always flagged via isDisambiguation).
mode	string	summary	summary (lead paragraph via REST summary), full (complete plain-text article → fullText), metadata (no text).
maxTextChars	integer	0	Truncate fullText on a word boundary (0 = whole article); fullTextChars keeps the real length.
includeSections	boolean	false	Table of contents [{level, title, chars}] (chars only in full mode; anchor in other modes).
includeCategories	boolean	false	Visible categories (≤100).
includeLinks	boolean	false	Titles of linked articles.
maxLinks	integer	50	1–500 cap for includeLinks.
includeLangLinks	boolean	false	languages (codes), languagesCount, langLinks [{lang, title, url}].
includeImages	boolean	false	Up to 20 images with URL and caption (thumbnail / originalImage are always included).
includeWikidata	boolean	false	Curated structured facts from the linked Wikidata item (see Reference → Wikidata properties). +3 requests per page (page data, the item, a batch of its labels per 50 linked items).
includePageviews	boolean	false	Human pageviews, all platforms, for the last pageviewsDays: total, avgPerDay, peakDay, peakViews.
pageviewsDays	integer	30	1–365, window ends yesterday (UTC).
includeDailyPageviews	boolean	false	Adds pageviews.daily [{date, views}].
dedupe	boolean	true	Save each page once even if several queries resolve to it.
fields	string[]	[]	Keep only these output fields, in this order. query and title are always kept (first, unless you list them). Letter case, spaces, - and _ do not matter (last_edited = lastEdited); unknown names are left out with a "did you mean" note in the status; a list in which no name can be in a row stops the run before any page is charged.

Reference

Allowed values (searchMode, mode)

• searchMode: text, title, nearmatch, exact.
• mode: summary, full, metadata.

These two fields take exactly these values; Apify refuses any other value before the run starts.

Language editions (lang)

Any live edition works. The codes below are built in; any other code (for example be-tarask, zh-min-nan, nds-nl) is checked with one request to <code>.wikipedia.org before the run reads anything, and a code that is not a Wikipedia edition stops the run with "did you mean" (only the start is charged). English names of the languages below and common native names (Deutsch, Русский, Қазақша…) are read as their codes; so are locale tags (en-US → en, pt_BR → pt, zh-Hant → zh).

Code	Language	Code	Language	Code	Language
en	English	de	German	fr	French
es	Spanish	it	Italian	pt	Portuguese
ru	Russian	ja	Japanese	zh	Chinese
ar	Arabic	nl	Dutch	pl	Polish
sv	Swedish	uk	Ukrainian	vi	Vietnamese
tr	Turkish	fa	Persian	ko	Korean
id	Indonesian	cs	Czech	he	Hebrew
hu	Hungarian	fi	Finnish	no / nn	Norwegian
da	Danish	ro	Romanian	el	Greek
bg	Bulgarian	sr	Serbian	hr	Croatian
sk	Slovak	sl	Slovenian	lt	Lithuanian
lv	Latvian	et	Estonian	th	Thai
hi	Hindi	bn	Bengali	ta	Tamil
te	Telugu	ml	Malayalam	mr	Marathi
ur	Urdu	ms	Malay	ca	Catalan
eu	Basque	gl	Galician	kk	Kazakh
uz	Uzbek	ky	Kyrgyz	tg	Tajik
tk	Turkmen	az	Azerbaijani	ka	Georgian
hy	Armenian	mn	Mongolian	be	Belarusian
tt	Tatar	ba	Bashkir	sq	Albanian
mk	Macedonian	bs	Bosnian	is	Icelandic
ga	Irish	cy	Welsh	af	Afrikaans
sw	Swahili	am	Amharic	ha	Hausa
yo	Yoruba	ne	Nepali	si	Sinhala
my	Burmese	km	Khmer	lo	Lao
tl	Tagalog	la	Latin	eo	Esperanto
simple	Simple English	zh-yue	Cantonese	ceb	Cebuano
war	Waray	arz	Egyptian Arabic	ckb	Central Kurdish
ku	Kurdish	ps	Pashto	sd	Sindhi
pa	Punjabi	gu	Gujarati	kn	Kannada
or	Odia	as	Assamese

Full list: https://meta.wikimedia.org/wiki/List_of_Wikipedias

Wikidata properties extracted (includeWikidata)

Output key	Property	Output key	Property
instanceOf	P31	country	P17
headquarters	P159	inception / dissolved	P571 / P576
foundedBy	P112	ceo / chairperson	P169 / P488
industry	P452	legalForm	P1454
ownedBy / parentOrganization	P127 / P749	subsidiaries	P355
employees ({value, year})	P1128	revenue / netProfit / totalAssets ({amount, currencyItem, year})	P2139 / P2295 / P2403
website	P856	coordinates	P625
tickerSymbol / stockExchange	P249 / P414	isin / lei	P946 / P1278
openCorporatesId / crunchbaseId	P1320 / P2088	rorId / gridId	P6782 / P2427
twitter / linkedinCompany / facebook / youtubeChannel	P2002 / P4264 / P2013 / P2397	image / logo	P18 / P154
population / capital	P1082 / P36	dateOfBirth / dateOfDeath	P569 / P570
citizenship / occupation / employer	P27 / P106 / P108	educatedAt / positionHeld / memberOf	P69 / P39 / P463
ownerOf	P1830	phone / email	P1329 / P968
streetAddress / postalCode	P6375 / P281	officialName / shortName	P1448 / P1813

Item values (people, places, companies, currencies) are returned as labels in the edition language, falling back to English, then Wikidata's language-neutral mul label (used by most person items since 2024), then any language. The original Q-ids are kept in wikidata.entityIds (e.g. { "ceo": "Q106028933", "foundedBy": ["Q483382", "Q332591", "Q19837"] }) for joins.

Preferred-rank claims win, deprecated ones are dropped, time-series claims (revenue, employees) return the latest value with its year. Item values are resolved to labels in lang, falling back to English.

Examples

KYB / due diligence on a list of companies

{ "queries": ["Kaspi.kz", "Air Astana", "Halyk Bank", "KazMunayGas"], "searchMode": "nearmatch", "includeWikidata": true, "includeLangLinks": true, "fields": ["query", "title", "description", "extract", "wikidata", "languagesCount", "url"] }

Brand attention tracker (run weekly)

{ "queries": ["Tesla, Inc.", "BYD Auto", "Rivian"], "searchMode": "exact", "mode": "metadata", "includePageviews": true, "pageviewsDays": 90, "includeDailyPageviews": true }

Full articles for a RAG index, with table of contents

{ "queries": ["Stablecoin", "Central bank digital currency", "Payment system"], "searchMode": "exact", "mode": "full", "includeSections": true, "includeCategories": true, "maxTextChars": 60000 }

Topic research in Russian — top 5 hits per query

{ "queries": ["агропромышленный комплекс Казахстана", "цифровизация сельского хозяйства"], "lang": "ru", "resultsPerQuery": 5, "skipDisambiguation": true }

Related-topics graph for content planning

{ "queries": ["Agentic AI"], "searchMode": "exact", "includeLinks": true, "maxLinks": 200, "includeCategories": true, "mode": "metadata" }

Output

One item per page found. Queries with no result and failed queries are not items and are not charged — they are listed in the SUMMARY record and the run's status message. Example (trimmed):

{
  "query": "Astana International Financial Centre",
  "rank": 1,
  "found": true,
  "lang": "en",
  "pageId": 57219556,
  "title": "Astana International Financial Centre",
  "requestedTitle": null,
  "description": "Financial hub in Astana, Kazakhstan",
  "isDisambiguation": false,
  "extract": "The Astana International Financial Centre (AIFC) is a financial hub in Astana, Kazakhstan …",
  "snippet": "The Astana International Financial Centre (AIFC) is a financial hub …",
  "thumbnail": "https://thumb.wikimedia.org/wikipedia/commons/thumb/…/330px-AIFC.jpg",
  "originalImage": "https://thumb.wikimedia.org/wikipedia/commons/thumb/…/960px-AIFC.jpg",
  "coordinates": { "lat": 51.09, "lon": 71.42 },
  "url": "https://en.wikipedia.org/wiki/Astana_International_Financial_Centre",
  "lastEdited": "2026-08-30T11:02:41Z",
  "revisionId": "1312345678",
  "wordCount": 1874,
  "wikidataId": "Q28155597",
  "wikidata": { "id": "Q28155597", "instanceOf": "financial centre", "country": "Kazakhstan", "inception": "2018-01-01", "headquarters": "Astana", "website": "https://aifc.kz", "twitter": "AIFC_KZ", "url": "https://www.wikidata.org/wiki/Q28155597", "entityIds": { "instanceOf": "Q338313", "country": "Q232", "headquarters": "Q1520" } },
  "pageviews": { "days": 30, "from": "2026-08-14", "to": "2026-09-12", "total": 2140, "avgPerDay": 71.3, "peakDay": "2026-09-02", "peakViews": 188 },
  "fetchedAt": "2026-09-13T08:05:12.345Z"
}

Field	Type	Meaning
query, rank, found	string, int, bool	Which input produced the row and its search rank.
lang, pageId, title, requestedTitle, redirected		Edition, stable page id, canonical title; requestedTitle when a redirect/search changed it.
description, extract, snippet	string	Wikidata short description, lead text, search snippet.
isDisambiguation	bool	Page is a disambiguation list.
thumbnail, originalImage, images[]		Lead image; images with includeImages.
coordinates	object	{lat, lon} for places.
url, lastEdited, revisionId, wordCount, pageLengthBytes		Page metadata: lastEdited is the time of the latest edit (the timestamp of revision revisionId, a string); wordCount comes from search (text / nearmatch); pageLengthBytes with mode: "full" or any of categories / links / other languages / Wikidata.
fullText, fullTextChars, sections[]		mode: "full" / includeSections.
categories[], links[], languages[], languagesCount, langLinks[]		Optional graph data.
wikidataId, wikidata{}		Q-id and curated facts (includeWikidata); item values as labels, their Q-ids in wikidata.entityIds.
pageviews{}		Attention metrics (includePageviews).
fetchedAt	ISO	Provenance.

A SUMMARY record in the key-value store: { lang, queries, mode, searchMode, saved, notFound (count), notFoundCount, notFoundInputs[], errors[{input, error}], errorCount, notReached, notReachedInputs[], duplicatesSkipped, disambiguationSkipped, notes[], stoppedEarly, stoppedBy, requests, throttledResponses, status } (lists hold the first 100). stoppedBy is limit (your max cost per run), timeout or aborted when the run ended early; notReachedInputs are the queries it did not finish.

Dataset views

The dataset has four table views: Pages (the default: title, description, extract, Wikidata id, last edit, URL), Full text (mode: "full": characters, sections, full text), Wikidata facts (includeWikidata: the facts object per page — values that hold several items, such as industries or founders, are lists, so they stay in one column) and Pageviews (includePageviews: views, per day, peak day and views, window). Views change only how the Console shows the data; JSON/CSV/Excel exports hold every field.

Use it from code / agents

curl -X POST "https://api.apify.com/v2/acts/yadroo~wikipedia-search/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"queries":["Kaspi.kz","Halyk Bank"],"searchMode":"nearmatch","includeWikidata":true}'

import { ApifyClient } from 'apify-client';
const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('yadroo/wikipedia-search').call({ queries: ['Stablecoin'], mode: 'full', includeSections: true });
const { items } = await client.dataset(run.defaultDatasetId).listItems();

from apify_client import ApifyClient
client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("yadroo/wikipedia-search").call(run_input={"queries": ["Tesla, Inc."], "searchMode": "exact", "includePageviews": True, "pageviewsDays": 90})
items = client.dataset(run["defaultDatasetId"]).list_items().items

MCP: add https://mcp.apify.com to Claude / Cursor / any MCP client and call the yadroo/wikipedia-search tool with the same JSON input.

Pricing

Pay per event: $0.001 per run start + $0.001 per page. Store discounts: Bronze −10 %, Silver −20 %, Gold and above −30 % on the page price; the start price is the same on every plan; platform usage is included. Typical runs: 10 company lookups with Wikidata ≈ $0.011; 500-name enrichment ≈ $0.5; full-article ingestion of 100 pages ≈ $0.1.

Limits & FAQ

• Rate limits — Wikimedia allows an identified client 200 requests per minute and asks for one request at a time. The actor sends requests one by one, starting them at least 350 ms apart across Wikipedia, Wikidata and the pageviews API (at most ~170 a minute), with a descriptive User-Agent. On HTTP 429/503 it waits out Retry-After (up to 60 s) or backs off (5 → 10 → 20 s); if Wikimedia keeps refusing, or asks for a longer wait, the run stops sending requests, keeps every page already saved and ends with "Stopped early — …" and the queries it did not reach in SUMMARY.notReachedInputs.

• How much fits in one run — each page costs one request for the summary, plus one per enabled block (mode: "full" or categories / links / other languages: one together; sections outside full mode, images and pageviews: one each; Wikidata: the page data request plus the item and one label request per 50 linked items); text, title and near-match search add one per query. Measured on 2 Oct 2026 (country and large-company articles), the default 15-minute run (900 s, of which the last 90 s are kept for saving) holds about:

  Input	Requests per query	Queries per default run
  searchMode: "exact", summary	1	~2,000 (0.37 s per page)
  text search, resultsPerQuery: 1, summary	2	~1,000
  mode: "full" + includeSections, text search	3	~550 (1.45 s per query)
  + includeWikidata, text search	5–6	~300 (2.7 s per query)
  + includeWikidata + includePageviews	6–7	~250

  For more, raise the run timeout (Run options → Timeout) or split the list. A run near its timeout stops starting new pages, keeps everything saved and ends SUCCEEDED with "Stopped before the run timeout: N rows saved"; the rest is listed in SUMMARY.notReachedInputs.

• Spending limit — the run never charges more than your "Maximum cost per run": it stops at the last page it may bill and says "Stopped at your spending limit: N rows delivered".

• Restarts — if Apify migrates or restarts the run, pages already saved are neither saved nor charged again, and finished queries are not asked again.

• Freshness — live from Wikipedia at run time; pageviews are published with ~1 day delay (window ends yesterday).

• Not found — a query with no page is not an item (you are not charged for it); it is listed in SUMMARY.notFoundInputs. A page or query that fails with an API error goes to SUMMARY.errors (not charged); the run fails only when no query could be answered at all.

• Disambiguation — flagged via isDisambiguation; use skipDisambiguation or searchMode: "text" with resultsPerQuery: 3 for ambiguous names.

• Text size — full articles can exceed 200 KB; use maxTextChars for LLM budgets.

• Wikidata coverage — only properties in the curated table are extracted; other claims are not returned (roadmap: wikidataProperties input for arbitrary P-ids).

• Roadmap — revision history / last editors, geosearch by coordinates, Wikipedia "on this day" and trending feeds.

---

Made by Yadroo. Sibling actors: openalex-works, arxiv-papers, openlibrary-books, domain-intel, sanctions-screen.