Wikipedia Articles Scraper (Apify Actor)

Fetch Wikipedia articles by title, free-text search, or category. Returns article summaries, full HTML with section index, search results, or category members — all in one unified JSON record shape. No auth, no proxy, no captcha. Powered by the public MediaWiki REST + Action API.

Source coverage

Endpoint	Auth	Captcha	Records	Use
https://en.wikipedia.org/api/rest_v1/page/summary/{title}	none	none	6.8M+ articles	summary mode
https://en.wikipedia.org/api/rest_v1/page/html/{title}	none	none	6.8M+ articles	full HTML mode
https://en.wikipedia.org/api/rest_v1/page/mobile-sections/{title}	none	none	6.8M+ articles	section index
https://en.wikipedia.org/w/api.php?action=query&list=search	none	none	6.8M+ articles	free-text search
https://en.wikipedia.org/w/api.php?action=query&list=categorymembers	none	none	1.6M+ categories	category members

All endpoints return JSON or HTML, no proxy / captcha / JS rendering needed, no auth, no API key. Wikipedia content is CC-BY-SA (freely reusable for non-commercial research; commercial reuse requires attribution).

What it does

• Four pipeline modes selectable via the mode input:
  • summary (default) — REST /page/summary/{title} for each pipe-separated title. Returns description, lead paragraph (extract), thumbnail, pageId, canonical URL.
  • full — REST /page/html/{title} (HTML body) + REST /page/mobile-sections/{title} (section index) for each title. Returns full HTML + structured sections[] with toclevel/line/anchor.
  • search — Action API list=search with titles as the free-text query, then fetch summary for each top hit. Returns ranked search results.
  • category — Action API list=categorymembers with Category:{titles}, then fetch full HTML for each top hit. Returns all members of a Wikipedia category.
• Titles are URL-encoded using the MediaWiki "percent-encoding for spaces → underscores" convention so "Albert Einstein" → "Albert_Einstein".
• Outputs are pushed to the default Apify dataset via Actor.pushData(...) per record.
• Smoke-tested inputs mode=summary, titles=Albert Einstein|Marie Curie|Ada Lovelace returned 3 non-null records on 2026-07-08.

Input

Field	Type	Default	Description
titles	textfield	`"Albert Einstein	Marie Curie
mode	select	"summary"	summary | full | search | category
maxItems	number (1-50)	5	Cap on records. For summary/full this caps pipe-separated titles. For search/category this caps top results fetched.

Output

One dataset record per article. See .actor/dataset_schema.json for the full shape. Sample (first record from a mode=summary, titles=Albert Einstein|Marie Curie|Ada Lovelace smoke run, abridged):

{
  "mode": "summary",
  "title": "Albert Einstein",
  "description": "German-born theoretical physicist (1879–1955)",
  "pageId": 736,
  "extract": "Albert Einstein (14 March 1879 – 18 April 1955) was a German-born theoretical physicist who is widely held to be one of the greatest and most influential scientists of all time. Best known for developing the theory of relativity, Einstein also made important contributions to the development of the theory of quantum mechanics.",
  "thumbnail": {
    "source": "https://upload.wikimedia.org/wikipedia/commons/thumb/3/3e/Einstein_1921_by_F_Schmutzer_-_restoration.jpg/320px-Einstein_1921_by_F_Schmutzer_-_restoration.jpg",
    "width": 320,
    "height": 415
  },
  "url": "https://en.wikipedia.org/wiki/Albert_Einstein",
  "fetchedAt": "2026-07-08T..."
}

For mode=full records additionally carry html (full article HTML) and sections[] (section index). For mode=search and mode=category records carry sourceTitle (the originating query or category) and rank (1-based result position).

Examples

Summaries for a curated list of physicists:

{
  "titles": "Albert Einstein|Marie Curie|Richard Feynman",
  "mode": "summary",
  "maxItems": 3
}

Full HTML + section index for one article (good for RAG ingestion):

{
  "titles": "Quantum mechanics",
  "mode": "full",
  "maxItems": 1
}

Free-text search + summary for each top hit (good for research):

{
  "titles": "machine learning transformer architecture",
  "mode": "search",
  "maxItems": 10
}

All articles in a Wikipedia category (good for corpus building):

{
  "titles": "Machine learning researchers",
  "mode": "category",
  "maxItems": 50
}

Limits / gotchas

• Polite-use: 1 request/second per endpoint, descriptive User-Agent. Wikipedia's API ToS asks for this; the actor sets User-Agent: apify-wikipedia-articles/0.1 automatically.
• maxItems is hard-capped at 50 (input schema enforces 1-50).
• Article titles with # or ? are URL-encoded by the actor before the API call; the title schema editor uses the canonical Wikipedia form (e.g. Albert Einstein, not Albert_Einstein).
• The full mode returns raw MediaWiki HTML which references Wikipedia-internal stylesheets and image URLs. Downstream consumers should rewrite image src to https://upload.wikimedia.org/... paths or strip styling for plain-text use cases.

License

Actor code: MIT (inherited from project template).

Data: Wikipedia content is CC-BY-SA 4.0; commercial reuse requires attribution to Wikipedia + the original editors.

Changelog

• 0.1.0 — initial release. Four modes (summary, full, search, category), unified record shape, hard cap 50.