Wikipedia Scraper — Article Content, Search & Metadata | from $0.50/1K

Built for RAG pipelines, LLM context builders, and knowledge-graph teams who need structured Wikipedia data without brittle HTML parsing. Powered by the official MediaWiki REST and Action APIs — no proxy, no auth, zero COGS.

Scrape Wikipedia articles, summaries, full-text extracts, categories, and metadata via the official MediaWiki REST and Action APIs. No proxy. No auth. Multi-language. Pay per article.

---

Features

• Title lookup — fetch any Wikipedia article by exact or near-exact title
• Full-text search — find articles by keyword; each result enriched with REST v1 summary
• Intro or full text — fullText toggle controls extract depth (intro paragraph vs. entire article)
• Categories — optional list of Wikipedia categories per article (up to 20)
• Multi-language — any Wikipedia language edition (language: "en", "de", "fr", "ja", etc.)
• Rich metadata — Wikidata Q-ID (wikibase_item), thumbnail URL, article byte length, last modified date
• parse_confidence — every record includes a quality score (1.0 = all fields present); zero silent failures
• $0 infrastructure cost — official APIs, no proxy, no auth tokens

---

Use cases

• Content research pipelines (LLM context, RAG datasets)
• Academic and news monitoring
• Knowledge graph enrichment (Wikidata cross-reference via wikibase_item)
• Education tools and topic discovery
• Multilingual dataset building

---

Input

Parameter	Type	Default	Description
titles	array	["Python (programming language)", "Machine learning"]	Article titles to fetch directly
searchQueries	array	[]	Full-text search terms; each enriched with REST summary
language	string	"en"	Wikipedia language code (en, de, fr, es, zh, ja, ru, …)
maxItems	integer	50	Total article cap (0 = unlimited)
maxSearchResults	integer	10	Results per search query (max 50)
fullText	boolean	false	Fetch full article text (can be large)
includeCategories	boolean	false	Include categories list (adds one API call per article)

---

Output schema

Each dataset row contains:

Field	Description
page_id	MediaWiki integer page ID
title	Canonical article title
summary	Intro paragraph (always populated)
extract	Full text if fullText=true, else same as summary
url	Desktop canonical article URL
lang	Wikipedia language code
description	Short Wikidata description (e.g. "German-born theoretical physicist")
wikibase_item	Wikidata Q-ID (e.g. "Q937") for cross-dataset enrichment
last_modified	ISO 8601 UTC of most recent edit
length	Article size in bytes
thumbnail_url	Lead image URL (when available)
categories	List of category names (prefix "Category:" stripped)
query	Title or search term that produced this record
source	"title_lookup" or "search"
scraped_at	ISO 8601 UTC for this run
parse_confidence	0.0–1.0 quality score; 1.0 = all fields present
warnings	Machine-readable list of missing/degraded field codes

---

Pricing

Pay-per-result (PPE): from $0.50 per 1,000 articles. No monthly subscription.

Volume	Cost
100 articles	~$0.05
1,000 articles	~$0.50
10,000 articles	~$5.00

Infrastructure cost to you: $0 — no proxy, no external API keys.

---

FAQ

Do I need a proxy or API key? No. The actor uses the official public MediaWiki APIs — no authentication, no proxy required.

What output formats are available? JSON (default), CSV, and Excel — downloadable from the Apify dataset UI or via the REST API.

Can I schedule this to run automatically? Yes. Use Apify's built-in scheduler to run on a cron schedule (hourly, daily, weekly) and push fresh data to your pipeline via webhooks.

What happens if an article title is not found or returns empty? The actor logs a warning and skips the record — no crash, no silent empty row. Failed titles are listed in the OUTPUT key at the end of the run.

---

API details

Built on two official, stable, zero-cost Wikipedia APIs:

• MediaWiki REST v1 — https://<lang>.wikipedia.org/api/rest_v1/page/summary/<title>
  Provides: intro extract, thumbnail, description, wikibase_item, timestamp, canonical URL.

• MediaWiki Action API — https://<lang>.wikipedia.org/w/api.php
  Provides: full-text extract, article length, categories, last revision.

Both APIs are public, documented, and maintained by the Wikimedia Foundation. No API key required.
This actor complies with Wikimedia API usage policy by sending a descriptive User-Agent.

---

Integrations

Built for RAG pipelines and LLM context builders pulling structured article text and metadata at scale — the JSON/dataset output drops into the tools you already run, no glue code:

• n8n / Make / Zapier — trigger a run or pipe every new dataset item into 500+ apps (Google Sheets, Airtable, Slack, HubSpot, your database) with no code: n8n, Make, Zapier.
• Webhooks — fire your own endpoint the moment a run finishes, to push results straight into your pipeline (docs).
• MCP server — expose this actor as a tool to Claude, Cursor, or any MCP client so an AI agent can pull this data mid-conversation (guide).
• API & SDKs — fetch the dataset as JSON, CSV, or Excel through the Apify REST API or the Python / JS SDKs.

See all Apify integrations.

Not affiliated with Wikipedia or the Wikimedia Foundation.

Data is sourced from public Wikipedia APIs under the Creative Commons Attribution-ShareAlike License.

---

Use with AI agents (MCP)

This actor is available as an MCP tool for Claude, GPT-4, and other AI agents that support the Model Context Protocol:

https://mcp.apify.com/?tools=bovi/wikipedia-scraper

Agents can call wikipedia-scraper to retrieve article content, summaries, and structured metadata on demand — ideal for RAG pipelines and knowledge enrichment tasks.

---

vs. competitors

	This actor	Typical Wikipedia scraper
Data source	Official MediaWiki REST + Action APIs	HTML scraping
Multi-language	✓ (any edition)	Usually EN only
Full-text extract	✓ (fullText toggle)	Rarely
parse_confidence	✓	No
Proxy needed	No	Often required
Price	from $0.50/1K	$3–10/1K

Usage statistics

This Actor creates a small, content-free summary at the end of each run. It is used only to monitor reliability and improve this Actor. A copy is saved as USAGE_STATS in your own Apify key-value store, so you can see the exact record created for your run.

Set disableUsageStats to true in the input to opt out. Nothing is sent then; your USAGE_STATS record only says that statistics were disabled.

Only these fields are recorded:

• schema version, Actor name and build number;
• UTC start and finish hour (not a precise timestamp);
• run duration, number of results and time to the first result, each as a coarse range;
• whether the result was empty, the end status, and an error type from a fixed list;
• memory setting and counts of charged events;
• names of the input fields you set, never their values;
• the selected option for input fields that offer a fixed list of choices (for example a sort order).

We do not collect input text, search terms, URLs, domains, usernames, email addresses, names, proxy credentials, tokens, scraped records, output items, raw error messages, stack traces, or hashes of any of those values. Records are kept for no longer than 13 months, used only as aggregated operational statistics, and never sold or shared.

Additional fields (Phase 2)

This Actor also records your Apify user ID, whether Apify marks the account as paying, the size range of list inputs, the selected country when the input offers a fixed list of countries, and one category from a fixed Actor taxonomy. We use these fields only for aggregate reliability, repeat-use and cross-Actor analysis; reports suppress any cell with fewer than five distinct users.

The same disableUsageStats: true input flag turns these fields off too. The user ID is removed after 13 months; we do not export, sell, share, or attempt to re-identify this data.

Run-outcome signals (v2)

To learn whether a run did what it was asked to do, the record also holds a few more coarse ranges and yes/no flags. None of them contains content:

• the result limit you asked for (a range, when the input has one) and what share of it was delivered;
• results delivered per input item you listed (a range);
• output quality as ranges: how fully the result fields were filled, the share of rows that look like errors, the share of duplicate rows, and how many different fields appeared. These are counted in memory while results are saved; no result content is kept;
• how the run was started (console, API, schedule, webhook, another Actor);
• how it ended: stopped by you, timed out, reached the requested limit, stopped by the charge limit, and how many times the platform moved the run;
• if this Actor reports it: how many items to process worked or failed (ranges) and one failure reason from a fixed list;
• a short code made from the names of the input fields you set, never their values.

Repeat-run fingerprint (v2)

When your Apify user ID is recorded (see above), the record also holds an 8-character one-way code made from your input (proxy settings left out) and this Actor's name. It only lets us see that the same account ran the same input again soon after an unsatisfying run; we never see the input itself. It is stored only in the database, never published, and reports use it in aggregate with the same five-user minimum. It is the one exception to the statement above that no hashes are collected, and disableUsageStats: true turns it off.