# Wikipedia Scraper (`solidcode/wikipedia-scraper`) Actor \[💰 $0.6 / 1K] Search Wikipedia or fetch exact articles by URL or title, and extract clean structured data — summaries, full plain text, categories, 30-day pageviews, thumbnails, coordinates, and language counts — across 300+ language editions. - **URL**: https://apify.com/solidcode/wikipedia-scraper.md - **Developed by:** [SolidCode](https://apify.com/solidcode) (community) - **Categories:** AI, Developer tools, Automation - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $0.60 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Wikipedia Scraper Turn any Wikipedia article into clean, structured data — summaries, full article text, categories, 30-day pageviews, Wikidata descriptions, thumbnails, geo-coordinates, and more, across 300+ language editions. Search by keyword or fetch exact articles you already know by URL or title. Built for researchers, data teams, and content & SEO analysts who need ready-to-use Wikipedia datasets without copy-pasting articles by hand or stitching together the raw API. ### Why This Scraper? - **300+ language editions** — the picker surfaces 40 of the largest editions (English, German, French, Spanish, Japanese, Chinese, Arabic, and more); paste any non-English Wikipedia URL and the actor scrapes that edition automatically. - **Two ways in: keyword search or exact fetch** — run a keyword search across an edition, *or* pull specific articles by full URL (`https://en.wikipedia.org/wiki/Alan_Turing`) or bare title ("Alan Turing"). Mix both in a single run. - **30-day pageview totals on every row** — a real popularity signal, not a guess: the trailing ~30-day view count for each article, ideal for ranking topics by actual reader demand. - **Sort by relevance or popularity** — order search hits by best textual match, or by most-referenced (incoming-link-weighted) to surface the canonical, authoritative article first. - **Full plain-text article body on demand** — toggle `fullText` to capture the entire article as clean plain text (not just the intro), ready for NLP, summarization, or training corpora. - **Wikidata one-line descriptions** — the short canonical descriptor (e.g. "English mathematician and computer scientist") pulled straight from Wikidata, perfect for tooltips and entity labels. - **Category taxonomy per article** — the full list of categories each article belongs to ("British computer scientists", "1912 births") for topic mapping and classification. - **Geo-coordinates for places & landmarks** — latitude/longitude on every article that has a location, so cities, monuments, and venues drop straight onto a map. - **Cross-edition reach signal** — `langCount` tells you in how many language editions an article exists, a quick indicator of global notability. ### Use Cases **Research & Academia** - Build structured corpora of articles on a topic for literature reviews and citation context - Compare how the same subject is covered across language editions - Track article size and last-edited dates to study how topics evolve - Rank subjects by 30-day readership to find what audiences actually care about **SEO & Content** - Pull authoritative summaries and Wikidata descriptions for entity-rich content - Identify high-traffic Wikipedia topics worth targeting in articles and FAQs - Map category taxonomies to plan topic clusters and internal linking - Surface the most-referenced canonical article for any keyword **Data Enrichment** - Enrich CRM and product records with one-line Wikidata descriptions - Add geo-coordinates to place names for mapping and location intelligence - Attach thumbnails and canonical URLs to people, companies, and landmarks - Resolve ambiguous names to the most popular matching article **Machine Learning & NLP** - Build full-text training datasets with clean plain-text article bodies - Generate summary/full-text pairs for summarization model fine-tuning - Create multilingual datasets by pulling the same topics across editions - Label corpora with category tags and word/byte-size metadata **Market & Competitive Intelligence** - Monitor pageview trends for brands, products, and public figures - Track which companies and technologies are gaining reader attention - Benchmark notability across markets using cross-edition coverage counts ### Getting Started #### Simple Keyword Search Search one edition and return the best-matching articles: ```json { "searchQueries": ["artificial intelligence"], "maxResultsPerSearch": 50 } ```` #### Fetch Exact Articles Pull specific articles by full URL or bare title — including non-English editions: ```json { "articleUrls": [ "https://en.wikipedia.org/wiki/Alan_Turing", "Marie Curie", "https://de.wikipedia.org/wiki/Albert_Einstein" ], "includeCategories": true } ``` #### Advanced — Popularity-Sorted Full-Text Dataset Search several topics, sort by popularity, and capture the full article body: ```json { "searchQueries": ["machine learning", "neural network", "deep learning"], "language": "en", "maxResultsPerSearch": 200, "sortBy": "popularity", "fullText": true, "includeCategories": true } ``` ### Input Reference #### What to Scrape | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `searchQueries` | string\[] | `[]` | Keywords to look up on Wikipedia (e.g. "climate change"). Each query runs its own search and returns the best-matching articles. Leave empty if you only want exact articles. | | `articleUrls` | string\[] | `[]` | Exact articles to fetch. Paste full Wikipedia URLs (e.g. `https://en.wikipedia.org/wiki/Alan_Turing`) or just a page title (e.g. "Alan Turing"). Full URLs set their own language automatically. | | `language` | select | `English` | Which Wikipedia edition to use for searches and bare titles. 40 common editions are listed; full URLs override this. | #### Results | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `maxResultsPerSearch` | integer | `50` | Maximum articles to return per search query (1–500). Articles fetched directly by URL or title are added on top. Recommended 50–200 for fast, affordable runs. | | `sortBy` | select | `Relevance (best match)` | Order search results. Options: "Relevance (best match)" or "Popularity (most referenced)". | #### Content | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `fullText` | boolean | `false` | Return the complete plain-text article body instead of just the intro summary. Richer data, larger dataset. | | `includeCategories` | boolean | `true` | Include the list of categories each article belongs to (e.g. "1912 births"). Helpful for classification and topic mapping. | ### Output Each article is one flat row. Here is a representative result: ```json { "pageId": 1208, "title": "Alan Turing", "language": "en", "url": "https://en.wikipedia.org/wiki/Alan_Turing", "summary": "Alan Mathison Turing was an English mathematician, computer scientist, logician, and cryptanalyst...", "fullText": "Alan Mathison Turing was an English mathematician... (full plain-text body when fullText is enabled)", "wikidataDescription": "English mathematician and computer scientist (1912–1954)", "categories": ["British computer scientists", "1912 births", "Alumni of King's College, Cambridge"], "thumbnail": "https://upload.wikimedia.org/wikipedia/commons/thumb/.../300px-Alan_Turing.jpg", "wordCount": 12840, "size": 198342, "pageviews": 415203, "langCount": 142, "coordinates": null, "lastEdited": "2026-06-10T08:14:32Z", "matchedQuery": "artificial intelligence", "scrapedAt": "2026-06-13T14:30:00Z" } ``` #### Core Fields | Field | Type | Description | |-------|------|-------------| | `pageId` | integer | Wikipedia page identifier | | `title` | string | Article title | | `language` | string | Language code of the edition this article came from | | `url` | string | Canonical article URL | | `matchedQuery` | string|null | The search query that surfaced this article (null for direct URL/title fetches) | | `scrapedAt` | string | ISO timestamp when the row was produced | #### Content | Field | Type | Description | |-------|------|-------------| | `summary` | string | Intro extract as clean plain text | | `fullText` | string|null | Full plain-text article body — populated only when `fullText` is enabled | | `wikidataDescription` | string|null | One-line canonical description from Wikidata | | `categories` | string\[] | Categories the article belongs to — populated when `includeCategories` is on | | `thumbnail` | string|null | Lead image URL (null when the article has no lead image) | #### Popularity & Metadata | Field | Type | Description | |-------|------|-------------| | `pageviews` | integer|null | Total reader views over the trailing ~30 days | | `langCount` | integer|null | Number of language editions this article exists in | | `wordCount` | integer | Word count of the returned text | | `size` | integer | Article size in bytes | | `lastEdited` | string | ISO timestamp of the most recent revision | #### Geo | Field | Type | Description | |-------|------|-------------| | `coordinates` | object|null | `{ "lat": …, "lon": … }` for articles with a location; null otherwise | ### Tips for Best Results - **Paste a non-English URL to reach any of the 300+ editions** — the language picker lists 40 common editions, but pasting `https://ja.wikipedia.org/wiki/...` or `https://fi.wikipedia.org/wiki/...` into `articleUrls` scrapes that edition directly, no picker needed. - **Leave `fullText` off for fast, cheap summary runs** — it pulls the entire article body and grows your dataset substantially. Turn it on only when you need full text for analysis or training. - **Sort by popularity to find the canonical article** — for ambiguous keywords, "Popularity (most referenced)" puts the authoritative, most-linked article first, ahead of niche or disambiguation pages. - **Use `pageviews` to rank topics by real demand** — it reflects actual 30-day readership, a far stronger signal than search rank for prioritizing content or research. - **Mix search and exact fetch in one run** — combine broad `searchQueries` with a hand-picked list of `articleUrls` to cover both discovery and known must-have articles. - **Start with 50 results per query to test** — confirm the data fits your needs, then raise `maxResultsPerSearch` (up to 500) for the full pull. ### Pricing **From $0.60 per 1,000 results** — flat pay-per-result, matching the lowest tier in this category while shipping more fields per article. Bronze, Silver, and Gold subscribers pay progressively less; the table below shows total cost at each discount tier. | Results | No discount | Bronze | Silver | Gold | |---------|-------------|--------|--------|------| | 100 | $0.072 | $0.068 | $0.064 | $0.060 | | 1,000 | $0.72 | $0.68 | $0.64 | $0.60 | | 10,000 | $7.20 | $6.80 | $6.40 | $6.00 | | 100,000 | $72.00 | $68.00 | $64.00 | $60.00 | A "result" is any article row in the output dataset. No compute or time-based charges — you pay per result, plus a small fixed per-run start fee. ### Integrations Export data in JSON, CSV, Excel, XML, or RSS. Connect to 1,500+ apps via: - **Zapier** / **Make** / **n8n** — Workflow automation - **Google Sheets** — Direct spreadsheet export - **Slack** / **Email** — Notifications on new results - **Webhooks** — Trigger custom APIs on run completion - **Apify API** — Full programmatic access ### Legal & Ethical Use This actor collects publicly available content from Wikipedia for legitimate research, analysis, and data enrichment. Wikipedia article text is published under the Creative Commons Attribution-ShareAlike (CC BY-SA) license — when you reuse or republish it, provide proper attribution and share derivative text under the same license. Users are responsible for complying with applicable laws and the Wikimedia Foundation's Terms of Use. Be respectful of the volunteer-run platform and use the data responsibly. # Actor input Schema ## `searchQueries` (type: `array`): Keywords to look up on Wikipedia, for example 'artificial intelligence' or 'climate change'. Each query runs its own search and returns the best-matching articles. Leave empty if you only want to fetch specific articles below. ## `articleUrls` (type: `array`): Fetch exact articles you already know. Paste full Wikipedia URLs (e.g. 'https://en.wikipedia.org/wiki/Alan\_Turing') or just the page title (e.g. 'Alan Turing'). When a full URL is used, its language is detected automatically; bare titles use the language selected below. ## `language` (type: `string`): Which Wikipedia edition to use for searches and for bare titles above. Full URLs override this automatically. ## `maxResultsPerSearch` (type: `integer`): Maximum number of articles to return for each search query (1-500; higher values are capped at 500). Articles fetched directly by URL or title are always included on top of this. Recommended 50-200 to keep runs fast and affordable. ## `sortBy` (type: `string`): How to order search results. Popularity favours articles with more incoming links and traffic. ## `fullText` (type: `boolean`): Return the complete plain-text body of each article instead of just the introductory summary. Useful for analysis and dataset building; increases dataset size. ## `includeCategories` (type: `boolean`): Include the list of categories each article belongs to (e.g. 'British computer scientists', '1912 births'). Helpful for classification and topic mapping. ## Actor input object example ```json { "searchQueries": [ "artificial intelligence" ], "articleUrls": [], "language": "en", "maxResultsPerSearch": 50, "sortBy": "relevance", "fullText": false, "includeCategories": true } ``` # Actor output Schema ## `overview` (type: `string`): Table of articles with title, language, summary, categories, pageviews, and link. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "searchQueries": [ "artificial intelligence" ], "articleUrls": [], "language": "en", "maxResultsPerSearch": 50, "sortBy": "relevance", "fullText": false, "includeCategories": true }; // Run the Actor and wait for it to finish const run = await client.actor("solidcode/wikipedia-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "searchQueries": ["artificial intelligence"], "articleUrls": [], "language": "en", "maxResultsPerSearch": 50, "sortBy": "relevance", "fullText": False, "includeCategories": True, } # Run the Actor and wait for it to finish run = client.actor("solidcode/wikipedia-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "searchQueries": [ "artificial intelligence" ], "articleUrls": [], "language": "en", "maxResultsPerSearch": 50, "sortBy": "relevance", "fullText": false, "includeCategories": true }' | apify call solidcode/wikipedia-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=solidcode/wikipedia-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Wikipedia Scraper", "description": "[💰 $0.6 / 1K] Search Wikipedia or fetch exact articles by URL or title, and extract clean structured data — summaries, full plain text, categories, 30-day pageviews, thumbnails, coordinates, and language counts — across 300+ language editions.", "version": "1.0", "x-build-id": "dR8H4sfMgNjkaoIvS" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/solidcode~wikipedia-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-solidcode-wikipedia-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/solidcode~wikipedia-scraper/runs": { "post": { "operationId": "runs-sync-solidcode-wikipedia-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/solidcode~wikipedia-scraper/run-sync": { "post": { "operationId": "run-sync-solidcode-wikipedia-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "searchQueries": { "title": "Search Queries", "type": "array", "description": "Keywords to look up on Wikipedia, for example 'artificial intelligence' or 'climate change'. Each query runs its own search and returns the best-matching articles. Leave empty if you only want to fetch specific articles below.", "items": { "type": "string" } }, "articleUrls": { "title": "Article URLs or Titles", "type": "array", "description": "Fetch exact articles you already know. Paste full Wikipedia URLs (e.g. 'https://en.wikipedia.org/wiki/Alan_Turing') or just the page title (e.g. 'Alan Turing'). When a full URL is used, its language is detected automatically; bare titles use the language selected below.", "items": { "type": "string" } }, "language": { "title": "Language", "enum": [ "en", "de", "fr", "es", "it", "pt", "nl", "pl", "ru", "uk", "sv", "no", "da", "fi", "cs", "sk", "hu", "ro", "bg", "el", "tr", "ar", "he", "fa", "hi", "id", "vi", "th", "ja", "ko", "zh", "ca", "eu", "gl", "sr", "hr", "et", "lt", "lv", "sl" ], "type": "string", "description": "Which Wikipedia edition to use for searches and for bare titles above. Full URLs override this automatically.", "default": "en" }, "maxResultsPerSearch": { "title": "Max Results Per Search", "minimum": 1, "maximum": 500, "type": "integer", "description": "Maximum number of articles to return for each search query (1-500; higher values are capped at 500). Articles fetched directly by URL or title are always included on top of this. Recommended 50-200 to keep runs fast and affordable.", "default": 50 }, "sortBy": { "title": "Sort Search Results By", "enum": [ "relevance", "popularity" ], "type": "string", "description": "How to order search results. Popularity favours articles with more incoming links and traffic.", "default": "relevance" }, "fullText": { "title": "Include Full Article Text", "type": "boolean", "description": "Return the complete plain-text body of each article instead of just the introductory summary. Useful for analysis and dataset building; increases dataset size.", "default": false }, "includeCategories": { "title": "Include Categories", "type": "boolean", "description": "Include the list of categories each article belongs to (e.g. 'British computer scientists', '1912 births'). Helpful for classification and topic mapping.", "default": true } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```