Wikipedia Scraper — Article Content, Search & Metadata | from $0.50/1K

Built for RAG pipelines, LLM context builders, and knowledge-graph teams who need structured Wikipedia data without brittle HTML parsing. Powered by the official MediaWiki REST and Action APIs — no proxy, no auth, zero COGS.

Scrape Wikipedia articles, summaries, full-text extracts, categories, and metadata via the official MediaWiki REST and Action APIs. No proxy. No auth. Multi-language. Pay per article.

---

Features

• Title lookup — fetch any Wikipedia article by exact or near-exact title
• Full-text search — find articles by keyword; each result enriched with REST v1 summary
• Intro or full text — fullText toggle controls extract depth (intro paragraph vs. entire article)
• Categories — optional list of Wikipedia categories per article (up to 20)
• Multi-language — any Wikipedia language edition (language: "en", "de", "fr", "ja", etc.)
• Rich metadata — Wikidata Q-ID (wikibase_item), thumbnail URL, article byte length, last modified date
• parse_confidence — every record includes a quality score (1.0 = all fields present); zero silent failures
• $0 infrastructure cost — official APIs, no proxy, no auth tokens

---

Use cases

• Content research pipelines (LLM context, RAG datasets)
• Academic and news monitoring
• Knowledge graph enrichment (Wikidata cross-reference via wikibase_item)
• Education tools and topic discovery
• Multilingual dataset building

---

Input

Parameter	Type	Default	Description
titles	array	["Python (programming language)", "Machine learning"]	Article titles to fetch directly
searchQueries	array	[]	Full-text search terms; each enriched with REST summary
language	string	"en"	Wikipedia language code (en, de, fr, es, zh, ja, ru, …)
maxItems	integer	50	Total article cap (0 = unlimited)
maxSearchResults	integer	10	Results per search query (max 50)
fullText	boolean	false	Fetch full article text (can be large)
includeCategories	boolean	false	Include categories list (adds one API call per article)

---

Output schema

Each dataset row contains:

Field	Description
page_id	MediaWiki integer page ID
title	Canonical article title
summary	Intro paragraph (always populated)
extract	Full text if fullText=true, else same as summary
url	Desktop canonical article URL
lang	Wikipedia language code
description	Short Wikidata description (e.g. "German-born theoretical physicist")
wikibase_item	Wikidata Q-ID (e.g. "Q937") for cross-dataset enrichment
last_modified	ISO 8601 UTC of most recent edit
length	Article size in bytes
thumbnail_url	Lead image URL (when available)
categories	List of category names (prefix "Category:" stripped)
query	Title or search term that produced this record
source	"title_lookup" or "search"
scraped_at	ISO 8601 UTC for this run
parse_confidence	0.0–1.0 quality score; 1.0 = all fields present
warnings	Machine-readable list of missing/degraded field codes

---

Pricing

Pay-per-result (PPE): from $0.50 per 1,000 articles. No monthly subscription.

Volume	Cost
100 articles	~$0.05
1,000 articles	~$0.50
10,000 articles	~$5.00

Infrastructure cost to you: $0 — no proxy, no external API keys.

---

FAQ

Do I need a proxy or API key? No. The actor uses the official public MediaWiki APIs — no authentication, no proxy required.

What output formats are available? JSON (default), CSV, and Excel — downloadable from the Apify dataset UI or via the REST API.

Can I schedule this to run automatically? Yes. Use Apify's built-in scheduler to run on a cron schedule (hourly, daily, weekly) and push fresh data to your pipeline via webhooks.

What happens if an article title is not found or returns empty? The actor logs a warning and skips the record — no crash, no silent empty row. Failed titles are listed in the OUTPUT key at the end of the run.

---

API details

Built on two official, stable, zero-cost Wikipedia APIs:

• MediaWiki REST v1 — https://<lang>.wikipedia.org/api/rest_v1/page/summary/<title>
  Provides: intro extract, thumbnail, description, wikibase_item, timestamp, canonical URL.

• MediaWiki Action API — https://<lang>.wikipedia.org/w/api.php
  Provides: full-text extract, article length, categories, last revision.

Both APIs are public, documented, and maintained by the Wikimedia Foundation. No API key required.
This actor complies with Wikimedia API usage policy by sending a descriptive User-Agent.

---

Integrations

Built for RAG pipelines and LLM context builders pulling structured article text and metadata at scale — the JSON/dataset output drops into the tools you already run, no glue code:

• n8n / Make / Zapier — trigger a run or pipe every new dataset item into 500+ apps (Google Sheets, Airtable, Slack, HubSpot, your database) with no code: n8n, Make, Zapier.
• Webhooks — fire your own endpoint the moment a run finishes, to push results straight into your pipeline (docs).
• MCP server — expose this actor as a tool to Claude, Cursor, or any MCP client so an AI agent can pull this data mid-conversation (guide).
• API & SDKs — fetch the dataset as JSON, CSV, or Excel through the Apify REST API or the Python / JS SDKs.

See all Apify integrations.

Not affiliated with Wikipedia or the Wikimedia Foundation.

Data is sourced from public Wikipedia APIs under the Creative Commons Attribution-ShareAlike License.

---

Use with AI agents (MCP)

This actor is available as an MCP tool for Claude, GPT-4, and other AI agents that support the Model Context Protocol:

https://mcp.apify.com/?tools=bovi/wikipedia-scraper

Agents can call wikipedia-scraper to retrieve article content, summaries, and structured metadata on demand — ideal for RAG pipelines and knowledge enrichment tasks.

---

vs. competitors

	This actor	Typical Wikipedia scraper
Data source	Official MediaWiki REST + Action APIs	HTML scraping
Multi-language	✓ (any edition)	Usually EN only
Full-text extract	✓ (fullText toggle)	Rarely
parse_confidence	✓	No
Proxy needed	No	Often required
Price	from $0.50/1K	$3–10/1K