Fandom Scraper

Fandom Scraper is an Apify Actor for people who need structured data from Fandom and Wikia wikis. It accepts exact article URLs, or a wiki plus a search query, category, or explicit page titles. Each scraped article becomes one dataset record with identity fields, a summary, infobox data, categories, images, links, section outline, and optional full content fields. The outcome is clean, machine-readable article data that works well in Apify runs, API workflows, and Apify MCP-based agent flows.

Best fit and connected workflows

This Actor fits workflows that start with a known Fandom page, a focused wiki query, or a category crawl on a single community site. It is a good match when you want article-level data rather than broad web crawling.

Typical connected workflows include:

• turning a single article URL into structured JSON for downstream analysis,
• searching a wiki for relevant pages and collecting the top matches,
• scraping all members of a category for a themed dataset,
• loading exact page titles when you already know the canonical pages,
• feeding article records into an AI agent that needs source-backed wiki facts through Apify MCP.

Practical scenario

Maya manages a lore knowledge base for a game project. She starts with the wiki subdomain harrypotter, enters elder wand in searchQuery, and keeps the default lean content settings. The run returns records with title, url, summary, infobox, categories, wordCount, and lastEdited. Maya uses the infobox and summary to confirm which page matches her reference topic, then sends the article URL to her editorial team for review.

Input fields

Field	Type	Purpose
startUrls	array of strings	Exact Fandom/Wikia article URLs. Takes priority over wiki query modes.
wiki	string	Wiki subdomain or host for search, category, and title-list modes.
searchQuery	string	Full-text search on the selected wiki.
category	string	Category name to scrape article members from.
titles	array of strings	Explicit list of article titles to scrape.
maxResults	integer	Upper bound for search or category results.
includeFullText	boolean	Adds contentText to each record.
includeHtml	boolean	Adds contentHtml to each record.
includeWikitext	boolean	Adds wikitext to each record.
maxLinks	integer	Caps included internal and external links per record.
proxyConfiguration	object	Proxy settings for the run.

Focused JSON example

{
  "wiki": "harrypotter",
  "searchQuery": "elder wand",
  "maxResults": 10,
  "includeFullText": false,
  "includeHtml": false,
  "includeWikitext": false,
  "maxLinks": 25
}

Output fields

Each dataset item represents one scraped Fandom article.

Field	Type	Purpose
wiki	string	Wiki host identifier.
subdomain	string	Wiki subdomain.
language	string	Wiki language code.
pageId	integer	MediaWiki page identifier.
title	string	Article title.
url	string	Canonical article URL.
namespace	integer	MediaWiki namespace number.
summary	string	Lead text extracted from the article.
infobox	object	Portable infobox data as structured fields.
infoboxImage	string	Main infobox image URL.
categories	array	Category names attached to the page.
images	array	Image filenames used on the page.
sections	array	Heading outline.
internalLinks	array	Internal page links found on the article.
externalLinks	array	External links found on the article.
wordCount	integer	Article length in words.
lastEdited	string	Last edit timestamp.
lastRevisionId	integer	Latest revision identifier.
contentText	string	Full plain-text body when enabled.
contentHtml	string	Rendered HTML body when enabled.
wikitext	string	Raw MediaWiki source when enabled.

Illustrative JSON record

{
  "wiki": "harrypotter.fandom.com",
  "subdomain": "harrypotter",
  "language": "en",
  "pageId": 12345,
  "title": "Wand",
  "url": "https://harrypotter.fandom.com/wiki/Wand",
  "namespace": 0,
  "summary": "A wand is a magical instrument used by witches and wizards to channel magical power.",
  "infobox": {
    "Manufacturer": "Ollivanders",
    "Usage": "Casting spells"
  },
  "infoboxImage": "https://static.wikia.nocookie.net/example/image.png",
  "categories": ["Wands"],
  "images": ["Example.png"],
  "sections": [
    {
      "level": 2,
      "title": "History",
      "anchor": "History"
    }
  ],
  "internalLinks": [],
  "externalLinks": [],
  "wordCount": 1200,
  "lastEdited": "2026-05-24T19:02:03Z",
  "lastRevisionId": 987654321
}

How it works

This Actor uses the public MediaWiki Action API over HTTP. The manifest indicates four input modes: exact article URLs, wiki titles, search, and category. The implementation also includes portable-infobox parsing, lean default records, and optional full text, HTML, and wikitext fields. By default, it uses Apify Proxy in datacenter mode, which is suitable for the MediaWiki API.

Pricing

Fandom Scraper uses Pay per event plus Apify platform usage. Charges are based on run start and on each article that is successfully scraped and pushed to the dataset. Open the live Pricing tab in Apify Console for the current event pricing and platform usage details.

Example: if a run starts once and returns twelve articles, the run is billed as one start event plus twelve article-scraped events.

Use with AI agents (MCP)

This Actor is available as an Apify Actor through Apify MCP. It is useful when an agent needs a precise tool for retrieving Fandom wiki pages as structured records.

Tool identity: khadinakbar/fandom-scraper

Use it when the agent needs article data from a known wiki, a search result set, a category listing, or a list of exact page titles. The output is a dataset of article records that an agent can interpret as source-backed wiki facts.

Scrape the Fandom wiki for harrypotter and return the top matches for elder wand. Include the summary and infobox fields, keep the output lean, and tell me which record looks like the canonical page.

Output interpretation:

• summary gives the lead text for quick reading,
• infobox captures structured page facts,
• categories, sections, images, and links help the agent understand page context,
• lastEdited and lastRevisionId provide provenance for the source snapshot.

Provenance and scope:

• records come from the selected Fandom wiki pages,
• each dataset item corresponds to one scraped article,
• startUrls takes priority when exact article URLs are supplied,
• maxResults bounds search and category scraping,
• maxLinks bounds the number of links included per record.

Cost guidance:

• use exact URLs or page titles for tightly scoped extraction,
• use search or category mode when discovery is needed,
• keep full text, HTML, and wikitext off unless the agent needs them.

Apify API example

import { ApifyClient } from "apify-client";

const client = new ApifyClient({
  token: process.env.APIFY_TOKEN,
});

const run = await client.actor("khadinakbar/fandom-scraper").call({
  wiki: "harrypotter",
  category: "Wands",
  maxResults: 5,
  includeFullText: false,
});

const datasetId = run.defaultDatasetId;
const { items } = await client.dataset(datasetId).listItems({ clean: true });

console.log(items);

Best results and outcome guidance

Use startUrls when you already know the exact article URLs. Use titles when you know the page names on a specific wiki. Use searchQuery for discovery and category for full membership collection. Keep includeFullText, includeHtml, and includeWikitext off for compact records, and raise maxLinks only when your workflow needs more outgoing link context.

Continue the workflow

• Then use Flashscore Scraper to extend Fandom Scraper with a neighboring entertainment data research source when the brief calls for Flashscore data.
• Then use IMDb Scraper for Titles, Cast, Ratings and Reviews to extend Fandom Scraper with a neighboring entertainment data research source when the brief calls for Imdb data.

Design note

I found that the dataset contract is intentionally broad: the overview view highlights title, url, summary, infobox, categories, wordCount, and lastEdited, while the full schema also includes pageId, sections, links, and optional content bodies. That made it clear the default record is meant to stay lean while still preserving room for deeper article analysis.

FAQ

Which Fandom pages does this Actor handle?
It works with Fandom and Wikia wiki articles, including pages addressed by exact URLs, titles, search results, and category members.

What input mode should I use for a known article?
Use startUrls for exact article URLs, or titles when you know the page names on a chosen wiki.

How should I approach discovery on a wiki?
Use searchQuery for keyword discovery or category for scraping a wiki category.

What information is included by default?
Default records include article identity, summary, infobox data, categories, word count, and edit metadata.

How can I get the full article body?
Enable includeFullText, includeHtml, or includeWikitext depending on whether you need plain text, rendered HTML, or raw wiki source.

Responsible use

Use this Actor for public Fandom and Wikia content that you are entitled to access and process. Review the source wiki's terms and attribution requirements before republishing or redistributing scraped material. Keep runs focused on the articles you need, and prefer lean outputs when a smaller record is sufficient for your workflow.