# Wikimedia Commons WikiProject Scraper (`parseforge/wikimedia-commons-wikiproject-scraper`) Actor Scrape Wikimedia Commons WikiProject pages: name, status, shortcut, participants, sections, categories, edit history. Export to CSV, JSON, Excel or XML. - **URL**: https://apify.com/parseforge/wikimedia-commons-wikiproject-scraper.md - **Developed by:** [ParseForge](https://apify.com/parseforge) (community) - **Categories:** Developer tools, Other - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $3.62 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. - **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use). - **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`. - **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`. - **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`. - **Any other language** — the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ![ParseForge Banner](https://github.com/ParseForge/apify-assets/blob/ad35ccc13ddd068b9d6cba33f323962e39aed5b2/banner.jpg?raw=true) ## 🗂️ Wikimedia Commons WikiProject Scraper > 🚀 **Export Wikimedia Commons WikiProject pages in seconds.** 20 fields per project, one row per page, up to 1,000,000 rows per run. Wikimedia Commons hosts hundreds of WikiProjects, the volunteer groups that curate media around a topic such as insects, aviation or chemistry. Their pages hold rosters, goals, section structure and edit history, but the site offers no export for them. This Actor reads each WikiProject page through the public Commons API and returns it as one flat row with 20 fields: image, name, status, shortcut, description, participants, sections, categories, creation and last-edit dates. Export to CSV, JSON, Excel, or XML. | 🎯 Target Audience | 💡 Primary Use Cases | |---|---| | Wikimedia community organizers | Audit which WikiProjects are active and who is on their rosters | | Digital archivists and GLAM staff | Map thematic coverage and find projects to partner with | | Open-knowledge researchers | Study volunteer coordination across projects over time | | Data journalists | Track when projects were founded and when they went quiet | ### 📋 What the Wikimedia Commons WikiProject Scraper does > 💡 **Why it matters:** WikiProject rosters and status are scattered across wikitext, subpages and hidden categories. This Actor normalizes all of it into one table. - 🔗 **Direct URLs:** paste any `Commons:WikiProject ...` page URLs; each becomes one row. Titles are matched case-insensitively, so `WikiProject_insects` still resolves to `WikiProject Insects`. - 🔍 **Topic search:** give search terms such as `birds` or `heraldry` and the Actor finds matching WikiProjects through the Commons search API, up to your max items. - 📚 **Full listing:** leave both empty and it lists WikiProjects alphabetically. - 👥 **Rosters from anywhere:** participants are read from `Participants` or `Members` sections and from `/Members` or `/Participants` subpages. - 🟢 **Status detection:** pages tagged historical or inactive come back as `Inactive`, everything else as `Active`. - 🕒 **Edit history:** creation date and creator, last edit date and editor, page size in bytes. - 🌐 **No browser, no login:** it uses the open MediaWiki API with a polite delay between calls; a proxy is available but off by default. ### 🎬 Full Demo (🚧 Coming soon) ### 📊 Output | Field | Description | |---|---| | 🖼 `imageUrl` | First image on the page as a direct `upload.wikimedia.org` URL, or `N/A` | | 📌 `projectName` | WikiProject name without the `Commons:` prefix | | 📄 `pageTitle` | Full page title on Wikimedia Commons | | 🔗 `url` | Canonical page URL | | 🆔 `pageId` | MediaWiki page id | | 🟢 `status` | `Active`, or `Inactive` when the page is marked historical or inactive | | ⚡ `shortcut` | `COM:` shortcuts declared on the page, or `Not Disclosed` | | 📝 `description` | First paragraph of the page as plain text, or `Not Disclosed` | | 👥 `participantCount` | Number of participants found | | 🧩 `sectionCount` | Number of section headings | | 📏 `contentLength` | Page size in bytes | | 🗓 `createdAt` | Timestamp of the first revision (ISO 8601) | | ✍️ `createdBy` | User who created the page | | 🕒 `lastEditedAt` | Timestamp of the latest revision (ISO 8601) | | ✏️ `lastEditedBy` | User of the latest revision | | 📑 `sections` | Section headings in page order | | 🙋 `participants` | User names listed as participants or members | | 🏷 `categories` | Categories of the page, hidden ones included | | 🕒 `scrapedAt` | When the row was collected | | ❌ `error` | `null` on success; a page that could not be read produces a row with only this field | Three real records from a run: ```json [ { "imageUrl": "https://upload.wikimedia.org/wikipedia/commons/6/67/System-file-manager-brown.svg", "projectName": "WikiProject Insects", "pageTitle": "Commons:WikiProject Insects", "url": "https://commons.wikimedia.org/wiki/Commons:WikiProject_Insects", "pageId": 495944, "status": "Inactive", "shortcut": "Not Disclosed", "description": "This WikiProject aims primarily to document in photograph, sound and video all species of Insecta.", "participantCount": 5, "sectionCount": 10, "contentLength": 4923, "createdAt": "2006-01-04T21:54:08Z", "createdBy": "TeunSpaans", "lastEditedAt": "2026-07-16T20:15:46Z", "lastEditedBy": "TenshiBot", "sections": [ "Article titles and common names", "Categories", "Status", "Quality", "Resources", "Requests", "Sister WikiProjects", "Participants", "Sample articles", "Cleanup" ], "participants": [ "TeunSpaans", "Svdmolen", "Kulac", "Keith Edkins", "Giancarlodessi" ], "categories": [ "Inactive Commons pages" ], "scrapedAt": "2026-09-08T01:11:55.043Z", "error": null }, { "imageUrl": "https://upload.wikimedia.org/wikipedia/commons/2/2e/Gnome-applications-science.svg", "projectName": "WikiProject Chemistry", "pageTitle": "Commons:WikiProject Chemistry", "url": "https://commons.wikimedia.org/wiki/Commons:WikiProject_Chemistry", "pageId": 1473569, "status": "Active", "shortcut": "COM:CHEM", "description": "This WikiProject is the Commons branch of the WikiProject Chemistry and other language Wikipedia workgroups (see interwiki links in the left column).", "participantCount": 0, "sectionCount": 2, "contentLength": 2554, "createdAt": "2006-12-18T17:24:45Z", "createdBy": "Benjah-bmm27", "lastEditedAt": "2023-11-03T16:05:50Z", "lastEditedBy": "Ameisenigel", "sections": [ "To-do list", "Quality assurance" ], "participants": [], "categories": [ "Commons WikiProjects", "Chemistry", "WikiProject Chemistry" ], "scrapedAt": "2026-09-08T01:11:56.458Z", "error": null }, { "imageUrl": "https://upload.wikimedia.org/wikipedia/commons/3/37/People_icon.svg", "projectName": "WikiProject Birds", "pageTitle": "Commons:WikiProject Birds", "url": "https://commons.wikimedia.org/wiki/Commons:WikiProject_Birds", "pageId": 1787159, "status": "Active", "shortcut": "Not Disclosed", "description": "This WikiProject descends from commons:WikiProject Tree of Life. It aims primarily to document in photograph, sound and video all species of birds.", "participantCount": 16, "sectionCount": 13, "contentLength": 5717, "createdAt": "2007-03-14T21:41:49Z", "createdBy": "Tony Wills", "lastEditedAt": "2024-11-07T12:29:20Z", "lastEditedBy": "Manojk", "sections": [ "Overview", "New to Commons?", "Fill the gaps!", "Categories", "Status", "Quality", "Resources", "Requests", "Identification", "Sister WikiProjects", "Participants", "Sample articles", "Cleanup" ], "participants": [ "Tony Wills", "Wsiegmund", "Anniolek", "Mindaugas Urbonas", "Dysmorodrepanis", "Tigershrike", "MeegsC", "Shyamal", "Dger", "innotata", "Kersti Nebelsiek", "Llywelyn2000", "Jcfidy", "Sharadapte", "सुबोध कुलकर्णी", "Manojk" ], "categories": [ "Commons WikiProjects", "WikiProject Tree of Life" ], "scrapedAt": "2026-09-08T01:11:58.988Z", "error": null } ] ``` A value the page does not provide comes back as `Not Disclosed` (shortcut, description) or `N/A` (image). Lists that are genuinely empty, such as a project without a roster, come back as `[]`. ### ✨ Why choose this Actor | | What you get | |---|---| | **API-backed, not screen-scraped** | Reads wikitext, revisions and categories through the MediaWiki API, so layout changes on the site do not break it. | | **One page, one row** | Every WikiProject becomes exactly one row with the same 20 keys, ready for a spreadsheet or a join on `pageId`. | | **Rosters that are actually complete** | Reads `{{User|...}}` templates and `[[User:...]]` links from sections and subpages; WikiProject Aviation returns its 80 members. | | **Three ways in** | Exact URLs, topic search, or an alphabetical listing when you give no input at all. | | **Polite by design** | Serial requests with a delay and a descriptive User-Agent, as Wikimedia's API etiquette asks. | ### 📈 How it compares to alternatives | Approach | What you get | What you give up | |---|---|---| | **This Actor** | Flat rows with rosters, status, dates and sections; search-based discovery; CSV/JSON/Excel/XML export | Media files themselves are not downloaded | | Manual browsing | Full context of each page | Hours of copy-paste for a handful of projects, no dates or counts | | Raw MediaWiki API calls | Everything, if you script it | Four requests per page, wikitext parsing, template and subpage handling written by you | | Generic Wikipedia scrapers | Article text | No WikiProject roster or status logic, no Commons namespace support | ### 🚀 How to use 1. [Create a free Apify account](https://console.apify.com/sign-up?fpr=vmoqkp); it comes with $5 of monthly credit. 2. Open the [Wikimedia Commons WikiProject Scraper](https://apify.com/parseforge/wikimedia-commons-wikiproject-scraper?fpr=vmoqkp). 3. Keep the five example URLs or paste your own WikiProject URLs, or type search terms such as `birds`. 4. Set **Max items** (free plans return up to 10 rows as a preview) and click **Start**. 5. Open the **Dataset** tab and export as CSV, Excel, JSON, or XML, or fetch it through the [Apify API](https://docs.apify.com/api/v2). Leave both URLs and search terms empty to list WikiProjects alphabetically. Tick **Include subpages** if you also want pages such as `Commons:WikiProject Aviation/Members` as their own rows. ### 💼 Business use cases #### 🤝 GLAM partnership scouting A museum searches `paintings`, `architecture` and `photography`, filters the rows to `status = Active`, and contacts the projects with the largest `participantCount` about a batch upload. #### 📊 Community health reporting A Wikimedia affiliate runs the full listing monthly and charts `lastEditedAt` and `participantCount` per project to spot WikiProjects that need attention. #### 🗺️ Coverage mapping An archive compares the `categories` and `sections` of every WikiProject against its own collection taxonomy to decide where its media will be found and curated. #### 🧑‍💻 Contributor outreach A program coordinator merges the `participants` lists of related projects to build an invite list for an edit-a-thon, with `createdBy` and `lastEditedBy` as first contacts. ### 🔌 Automating Wikimedia Commons WikiProject Scraper - **Make / Zapier:** trigger a run on a schedule and append new rows to a sheet or CRM. - **Slack:** post a message when a tracked project flips from `Active` to `Inactive`. - **Airbyte:** load the dataset into a warehouse and join on `pageId` across runs. - **GitHub Actions:** run the Actor in CI and commit the CSV to a data repository. - **Google Drive:** export to Excel and drop the file into a shared folder after every run. ### 🌟 Beyond business use cases - **Research:** longitudinal studies of volunteer coordination on Commons using `createdAt`, `lastEditedAt` and roster sizes. - **Personal:** find the right WikiProject to join for your hobby and see who is active there. - **Non-profit:** document which topics have organized curation and which are gaps. - **Experimentation:** feed `description` and `sections` into a classifier to cluster projects by scope. ### 🤖 Ask an AI assistant about this scraper Give an AI agent live access to the Actor through the Model Context Protocol: ```bash claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/wikimedia-commons-wikiproject-scraper" ``` Then ask in plain language, for example "list the active bird-related WikiProjects on Commons with their participant counts". ### ❓ Frequently Asked Questions #### ❓ What is a Wikimedia Commons WikiProject? A group of contributors who organize media curation around a topic. Each project lives at `Commons:WikiProject ` and typically lists goals, participants, tasks and categories. #### ❓ Do I need a Wikimedia account or API key? No. The Actor reads the public MediaWiki API without authentication. #### ❓ How do I get a specific project? Paste its URL as a start URL. Capitalization after the first letter is forgiven: `WikiProject_insects` resolves to `WikiProject Insects`. #### ❓ How do search terms work? Each term runs a Commons search restricted to WikiProject titles. Matching projects are scraped in relevance order until **Max items** is reached. Search terms take precedence over start URLs. #### ❓ Why is `participants` empty for some projects? Some projects, for example WikiProject Chemistry, have no roster on the page or on a `/Participants` or `/Members` subpage. The row is still returned with `participantCount: 0`. #### ❓ What does `status: Inactive` mean? The page carries a historical or inactive tag or sits in the "Inactive Commons pages" category. Everything else is reported as `Active`. #### ❓ Where does `imageUrl` come from? The first `[[File:...]]` in the page wikitext, or the first image the page renders, turned into a direct `upload.wikimedia.org` URL. Pages without images return `N/A`. #### ❓ Are subpages included? Not by default: one row is one WikiProject. Turn on **Include subpages** to also return pages such as `Commons:WikiProject Aviation/Members` from search or the listing. #### ❓ Does it download media files? No. It extracts metadata and text about the project page only. #### ❓ Is there a rate limit? The Actor makes three to four API calls per page in series with a short pause, in line with Wikimedia's API etiquette. Use **Max items** to bound the run. #### ❓ Can I run it on a free plan? Yes. Free plans return up to 10 rows per run as a preview; paid plans return up to 1,000,000. #### ❓ What happens if a page does not exist? That page produces a row containing only an `error` field with the reason; the other pages are unaffected. ### 🔌 Integrate with any app Run the Actor from the [Apify API](https://docs.apify.com/api/v2) or the [JavaScript and Python clients](https://docs.apify.com/api/client/js), schedule it in the Apify Console, and connect the dataset to Make, Zapier, Airbyte, Google Sheets, Slack or any webhook target. ### 🔗 Recommended Actors - [Wikimedia Commons Category Hierarchy Scraper](https://apify.com/parseforge/wikimedia-commons-category-scraper?fpr=vmoqkp): walk a category tree and its subcategories. - [Wikimedia Commons Media Scraper](https://apify.com/parseforge/wikimedia-commons-media-scraper?fpr=vmoqkp): file metadata, licenses and download URLs. - [Wikimedia Commons Users Scraper](https://apify.com/parseforge/wikimedia-commons-users-scraper?fpr=vmoqkp): profiles and edit counts for the participants you find here. - [Wikimedia Commons Page History Scraper](https://apify.com/parseforge/wikimedia-commons-history-scraper?fpr=vmoqkp): full revision history of any page. > 💡 **Pro Tip:** browse the complete [ParseForge collection](https://apify.com/parseforge?fpr=vmoqkp) for more data sources. **🆘 Need Help?** [Open our contact form](https://tally.so/r/BzdKgA) with your run ID, your input, and what you expected. > **⚠️ Disclaimer:** This Actor is an independent tool and is not affiliated with, endorsed by, or sponsored by the Wikimedia Foundation. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of use and applicable data-protection laws, including GDPR, CCPA, and PIPL. # Actor input Schema ## `startUrls` (type: `array`): WikiProject page URLs on Wikimedia Commons, one row per page, e.g. https://commons.wikimedia.org/wiki/Commons:WikiProject\_Insects. Leave both Start URLs and Search terms empty to list WikiProjects alphabetically. ## `searchTerms` (type: `array`): Find WikiProjects by topic (e.g. birds, heraldry, Argentina) through the Commons search API. Takes precedence over Start URLs. Each matching project becomes one row, up to Max items. ## `maxItems` (type: `integer`): Free users: Limited to 10 items (preview). Paid users: Optional, max 1,000,000 ## `includeSubpages` (type: `boolean`): Also return subpages such as Commons:WikiProject Aviation/Members. Off by default so one row is one WikiProject. ## `proxyConfiguration` (type: `object`): Off by default. Switch on Apify Proxy for high-volume runs. ## Actor input object example ```json { "startUrls": [ { "url": "https://commons.wikimedia.org/wiki/Commons:WikiProject_Insects" }, { "url": "https://commons.wikimedia.org/wiki/Commons:WikiProject_Chemistry" }, { "url": "https://commons.wikimedia.org/wiki/Commons:WikiProject_Aviation" }, { "url": "https://commons.wikimedia.org/wiki/Commons:WikiProject_Birds" }, { "url": "https://commons.wikimedia.org/wiki/Commons:WikiProject_Mammals" } ], "maxItems": 10, "includeSubpages": false, "proxyConfiguration": { "useApifyProxy": false } } ``` # Actor output Schema ## `overview` (type: `string`): Key fields of every WikiProject page ## `fullData` (type: `string`): Complete dataset with all 20 fields # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "startUrls": [ { "url": "https://commons.wikimedia.org/wiki/Commons:WikiProject_Insects" }, { "url": "https://commons.wikimedia.org/wiki/Commons:WikiProject_Chemistry" }, { "url": "https://commons.wikimedia.org/wiki/Commons:WikiProject_Aviation" }, { "url": "https://commons.wikimedia.org/wiki/Commons:WikiProject_Birds" }, { "url": "https://commons.wikimedia.org/wiki/Commons:WikiProject_Mammals" } ], "maxItems": 10 }; // Run the Actor and wait for it to finish const run = await client.actor("parseforge/wikimedia-commons-wikiproject-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "startUrls": [ { "url": "https://commons.wikimedia.org/wiki/Commons:WikiProject_Insects" }, { "url": "https://commons.wikimedia.org/wiki/Commons:WikiProject_Chemistry" }, { "url": "https://commons.wikimedia.org/wiki/Commons:WikiProject_Aviation" }, { "url": "https://commons.wikimedia.org/wiki/Commons:WikiProject_Birds" }, { "url": "https://commons.wikimedia.org/wiki/Commons:WikiProject_Mammals" }, ], "maxItems": 10, } # Run the Actor and wait for it to finish run = client.actor("parseforge/wikimedia-commons-wikiproject-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrls": [ { "url": "https://commons.wikimedia.org/wiki/Commons:WikiProject_Insects" }, { "url": "https://commons.wikimedia.org/wiki/Commons:WikiProject_Chemistry" }, { "url": "https://commons.wikimedia.org/wiki/Commons:WikiProject_Aviation" }, { "url": "https://commons.wikimedia.org/wiki/Commons:WikiProject_Birds" }, { "url": "https://commons.wikimedia.org/wiki/Commons:WikiProject_Mammals" } ], "maxItems": 10 }' | apify call parseforge/wikimedia-commons-wikiproject-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,parseforge/wikimedia-commons-wikiproject-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/vXaPv1gzPxCercPQv/builds/B1x9BUvrIvkrrKbHj/openapi.json