MediaWiki Scraper - Wiki Pages, Text & Categories

Scrape any MediaWiki or Fandom wiki in bulk. Extract page title, URL, intro text, categories, byte length, last edit date, revision ID and redirect status to CSV/JSON. Works on any wiki running MediaWiki. No API key.

---

What does the MediaWiki Scraper do?

This Actor turns MediaWiki data into a structured dataset. You point it at a whole wiki or a search within a wiki, it walks the result pages one after another, and it writes one clean row per page into your dataset — ready to export as JSON, CSV or Excel, or to pull straight from the Apify API.

Every MediaWiki installation — Fandom's quarter of a million wikis, gaming and software wikis, corporate knowledge bases — answers the same api.php query interface. The Actor uses it with a page generator and follows MediaWiki's own continue token, which is the only way to walk a wiki larger than a single batch without skipping or repeating pages. Pagination is followed automatically until it runs out of results, hits your page limit or hits your Max items cap, whichever comes first. Every row is de-duplicated across the whole run, so you are never billed twice for the same page.

There is no API key, no login and no browser involved. That keeps runs fast and cheap, and it means you can schedule the Actor without worrying about credentials expiring.

Who is it for?

• AI and NLP teams building domain text corpora from community wikis.
• Game and media researchers extracting structured lore or item data.
• Knowledge-management teams auditing an internal MediaWiki.
• Developers mirroring a wiki into a search index or app.
• Analysts measuring how active and how large a wiki is.

Use cases

• Export every page on a wiki with its intro text and categories.
• Search a wiki for a term and export only the matching pages.
• Find the largest or most recently edited pages on a wiki.
• Build a domain-specific corpus for retrieval or fine-tuning.
• Map a wiki's category structure across thousands of pages.

Why use this MediaWiki Scraper?

• 🔑 Keyless — no account, no API token, no cookies to paste.
• 📦 18 fields per page — everything the source exposes, already typed.
• 📄 Real pagination — it walks page after page instead of returning the first screen.
• 🎯 Precise caps — Max items stops the run exactly where you want it, so the bill is predictable.
• 📊 Export anywhere — JSON, CSV, Excel or HTML, plus the Apify API and integrations.
• 💸 Pay per result — you pay for rows you actually receive, with no platform fees to calculate.

What data can you extract?

Every run produces one row per page, with these fields:

Field	Type	Description
wikiDomain	string	Domain of the wiki this row came from
pageId	string	MediaWiki page ID
title	string	Page title
pageUrl	string	Canonical page URL
editUrl	string	URL of the page's edit view
namespace	number	MediaWiki namespace number, 0 is the main article space
extract	string	Plain-text intro of the page when the wiki provides extracts
categories	string	Category names, separated by a pipe
categoryCount	number	Number of categories on the page
lengthBytes	number	Page size in bytes
lastRevisionId	string	ID of the most recent revision
lastEditedAt	string	ISO timestamp the page was last touched
isRedirect	boolean	Whether the page is a redirect
contentModel	string	Content model, normally wikitext
pageLanguage	string	Language code of the page
query	string	Search term this row came from, or the whole wiki
page	number	Continuation round the page appeared in
scrapedAt	string	ISO timestamp of extraction

Output example

{
  "wikiDomain": "minecraft.fandom.com",
  "pageId": "87329",
  "title": "Advancement",
  "pageUrl": "https://minecraft.fandom.com/wiki/Advancement",
  "editUrl": "https://minecraft.fandom.com/wiki/Advancement?action=edit",
  "namespace": 0,
  "extract": "Advancements are a way to gradually guide new players into Minecraft and give them challenges to complete, similar to the basic system of achievements in Bedrock Edition.",
  "categories": "Data packs | Gameplay | Java Edition | Pages with Java sound tables | Pages with broken file links | Pages with missing sprites | Pages with sound tables | Upcoming",
  "categoryCount": 8,
  "lengthBytes": 75931,
  "lastRevisionId": "2400199",
  "lastEditedAt": "2026-09-11T23:01:55Z",
  "isRedirect": false,
  "contentModel": "wikitext",
  "pageLanguage": "en",
  "query": "whole wiki",
  "page": 3,
  "scrapedAt": "2026-09-19T11:22:03.517Z"
}

How to use

Option A — a whole wiki

{
  "includeExtract": true,
  "maxItems": 2000,
  "maxRoundsPerWiki": 40,
  "skipRedirects": true,
  "wikiUrls": [
    "https://minecraft.fandom.com"
  ]
}

1. Open the Actor and fill in the first field.
2. Set the page limit and Max items to bound the run.
3. Click Start, then export from the Output tab.

Option B — a search within a wiki

{
  "includeExtract": true,
  "maxItems": 500,
  "maxRoundsPerWiki": 20,
  "searchTerm": "redstone",
  "wikiUrls": [
    "https://minecraft.fandom.com"
  ]
}

Everything in the first field is processed independently, so you can batch several targets into one run and split them apart afterwards.

Input parameters

Parameter	Type	Default	Description
wikiUrls	array	–	Home pages of the wikis to scrape, e.
searchTerm	string	–	Use the wiki's own search instead of walking every page.
namespace	integer	0	MediaWiki namespace to read.
skipRedirects	boolean	true	Ignore redirect stubs and return only real content pages.
includeExtract	boolean	true	Fetch each page's plain-text intro.
maxRoundsPerWiki	integer	20	How many result pages to walk for each wiki.
maxItems	integer	500	Stop after this many pages.
maxConcurrency	integer	2	Parallel requests.
proxyConfiguration	object	{"useApifyProxy": true}	Proxy used to fetch pages.

Tips for best results

• Paste the wiki's home page, e.g. https://minecraft.fandom.com — the Actor finds api.php itself.
• Leave the search term empty to walk every page in the namespace in alphabetical order.
• Add a search term to use the wiki's own search instead of a full walk.
• Namespace 0 is articles; 14 is categories, 10 is templates — leave it at 0 for content.
• The API returns up to 50 pages per round for anonymous callers, and the Actor follows the continue token automatically.
• extract needs the TextExtracts extension, which Fandom and Wikipedia have but a bare MediaWiki may not; it comes back empty there.
• isRedirect true means the page is a pointer, not content — filter it out for a corpus.
• lengthBytes is the quickest way to separate stubs from substantial articles.
• Keep concurrency low — many wikis are volunteer-run on small servers.
• Pair with the Wikipedia scrapers when you need revision history rather than current text.

Integrations

Send results straight into the tools you already use: Google Sheets, Slack, Zapier, Make, Airtable or any Webhook. You can also schedule the Actor to run hourly, daily or weekly and have each run append to the same dataset, which is how you build a price or availability history rather than a one-off snapshot.

API usage

Run the Actor and collect results from any language. Replace <YOUR_TOKEN> with your Apify API token.

cURL

curl -X POST "https://api.apify.com/v2/acts/logiover~mediawiki-wiki-scraper/run-sync-get-dataset-items?token=<YOUR_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"includeExtract": true, "maxItems": 2000, "maxRoundsPerWiki": 40, "skipRedirects": true, "wikiUrls": ["https://minecraft.fandom.com"]}'

Node.js

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: '<YOUR_TOKEN>' });
const run = await client.actor('logiover/mediawiki-wiki-scraper').call({"includeExtract": true, "maxItems": 2000, "maxRoundsPerWiki": 40, "skipRedirects": true, "wikiUrls": ["https://minecraft.fandom.com"]});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Python

from apify_client import ApifyClient

client = ApifyClient('<YOUR_TOKEN>')
run = client.actor('logiover/mediawiki-wiki-scraper').call(run_input={"includeExtract": true, "maxItems": 2000, "maxRoundsPerWiki": 40, "skipRedirects": true, "wikiUrls": ["https://minecraft.fandom.com"]})
for item in client.dataset(run['defaultDatasetId']).iterate_items():
    print(item)

Use with AI agents (MCP)

This Actor is available through the Apify MCP server, so an AI agent can call it as a tool. Point your agent at https://mcp.apify.com and it can run the MediaWiki Scraper on demand — for example: "Pull the first 500 pages from MediaWiki and summarise what you find." The agent receives the same structured rows you would get from the UI.

FAQ

Do I need a MediaWiki account or API key?

No. The Actor reads publicly available data only. There is nothing to authenticate and no credentials to rotate.

How many pages can I get in one run?

As many as the source exposes. Raise the page limit and Max items together; the run stops at whichever limit it reaches first.

Why did I get fewer rows than I asked for?

The source ran out of pages. That is normal for narrow searches — broaden the query or add more targets to one run.

Are results de-duplicated?

Yes. Each page is emitted once per run, even when it appears on several pages, so you are never billed twice for the same record.

Why are some fields empty?

MediaWiki does not publish every attribute for every page. Empty means the source did not supply it, not that extraction failed.

What export formats are supported?

JSON, CSV, Excel, HTML and RSS from the Output tab, plus the Apify API and any integration you connect.

How fast is it?

It is pure HTTP with no browser, so a page of results typically takes a second or two. Raise Max concurrency carefully — the source rate-limits aggressive crawling.

Can I schedule it?

Yes. Use the Apify scheduler to run it on any interval and append each run to the same dataset for time-series analysis.

Does it work behind a proxy?

It uses Apify Proxy automatically. You can switch groups or supply your own proxies in Proxy configuration.

How often does the data change?

MediaWiki updates continuously. Re-run whenever you need current data; the Actor always reads the live source, never a cache.

Is the output schema stable?

Yes. Field names and types are fixed, so downstream pipelines will not break between runs.

What if the source changes its format?

Open an issue on the Issues tab and it gets fixed. The Actor is actively maintained.

Is it legal?

This Actor reads only publicly available data from MediaWiki — the same content any visitor or client sees without logging in. It does not bypass authentication, does not collect private data and does not attempt to defeat access controls. You are responsible for how you use the output: respect the source's terms of service, applicable copyright, and data-protection law such as GDPR where personal data is involved. Scraping public data is generally lawful in the EU and the US, but the responsibility for the downstream use of that data sits with you.

Related scrapers

• Wikipedia Category Scraper — Wikipedia specifically
• Wikipedia Revision History — Edit history
• Wikidata Scraper — Structured claims
• Website Text & Markdown — Non-wiki sites