Wikimedia Commons Village Pump Scraper

Wikimedia Commons Village Pump Scraper

Scrape Wikimedia Commons Village Pump discussions via the MediaWiki API, up to a million revisions per run. Each result includes revision author, timestamp, comment (containing section title), and size delta. No login or API key. Export to CSV, JSON, Excel, or XML.

Wikimedia Commons Village Pump is the central forum for policy, technical, and community discussions at the Wikimedia Commons. Its revision history is publicly accessible via the standard MediaWiki API, but extracting structured data from it manually is tedious. This Actor reads that API directly, returns every revision as a row, and optionally filters by search term to isolate discussions on a given topic.

Who uses it	What they scrape Wikimedia Commons Village Pump for
Wikimedia Commons moderators and administrators	Track which users are active in which discussion sections over time.
Open-data researchers	Analyze the frequency and sentiment of community discussions on specific policies.
GLAM (Galleries, Libraries, Archives, Museums) institutions	Monitor discussions about their uploads and content inclusion.
Data journalists	Find trending topics and key participants in the Commons community.

What it does

This Actor collects the revision history of the Wikimedia Commons Village Pump page (and its archives) via the MediaWiki API, and returns each revision as a flat row with user, timestamp, comment, and size.

• 🔍 Search filter: enter a search term to return only revisions whose comment (section title or edit summary) contains that term.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

What you can do with Wikimedia Commons Village Pump data

🔍 Trace policy discussion trends.

A Commons administrator runs it weekly to collect all Village Pump revision comments containing 'copyright' to see how the policy debate evolves.

📈 Identify most active contributors.

A community manager pulls the full revision list, counts distinct users per month, and publishes a participation heatmap.

🗂️ Archive discussions for offline analysis.

A researcher downloads every revision from the last five years into a CSV to study the issue lifecycle of Commons community decisions.

📋 Monitor mentions of GLAM uploads.

A museum curator searches for 'Wiki Loves Monuments' and gets a timeline of discussions affecting their institution's content.

Why choose this scraper

	What you get
Revision data	Each row is one revision: author, timestamp, comment, size delta.
API-sourced	Data comes from the verified Wikimedia API - no HTML scraping, no anti-bot blocks.
Filter by topic	Use a search term to isolate discussions on a specific subject.
Export flexible	Results in CSV, JSON, Excel, XML - ready for analysis.

What a Wikimedia Commons Village Pump record looks like

Every record returns as one flat JSON row. Here is a real one from a run:

{
 "pageid": 751116,
 "ns": 4,
 "title": "Commons:Village pump",
 "user": "SpBot",
 "timestamp": "2026-08-28T03:07:11Z",
 "size": 215168,
 "comment": "archive 1 section: 1 to [[Commons:Village pump/Archive/2026/08]] (after section [[Commons:Village pump/Archive/2026/08#\"Information_and_education_only\"|\"Information_and_education_only\"]]) - previous edit: [[:User:JWilz12345|JWilz12345]], 2026-08-28 02:20",
 "isNewSection": false,
 "isReply": false,
 "isBot": true,
 "scrapedAt": "2026-08-28T04:11:56.728Z"
}

Every value above comes from a real run. A field a record does not have comes back as null.

Configure the run

Drive the Actor by setting a maximum number of revisions to collect. Optionally provide a search term to filter revisions by comment content as they are fetched, so only matching discussions reach your dataset. The Input tab lists every parameter.

A first run with the defaults:

{
 "maxItems": 10
}

A larger pull:

{
 "maxItems": 200
}

Free users

Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.

Run it

1. Create a free Apify account with $5 in credit.
2. Open the Wikimedia Commons Village Pump Scraper.
3. Set your inputs and any filters, then click Start.
4. Export the results as CSV, Excel, JSON, or XML from the Dataset tab.

Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.

Use with AI agents (MCP)

Give an AI agent live access to Wikimedia Commons Village Pump through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

$claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/wikimedia-commons-village-pump-scraper"

Then prompt it in plain language to run the scraper and read back the results.

Troubleshooting

Why am I getting no results?

Check that the Village Pump page exists and the URL is correct. Also check if your searchTerm is too specific - try without it.

The data seems incomplete - some discussions are missing.

This Actor only fetches the main Village Pump page at present. Archived discussions are not included unless we add support. You may need to run the Actor on individual archive page URLs in a future version.

I get an error like 'API limit reached'.

The MediaWiki API has rate limits for anonymous requests. The Actor handles these automatically with retries and delays, but setting maxItems too high at once may trigger them. Try a lower value or run multiple smaller batches.

The 'searchTerm' filter is not working as expected.

The search is applied on the revision comment field only. Because the comment often includes the section title and edit summary, a broad term like 'policy' works better than a specific phrase. Check for typos and case sensitivity - it is case-sensitive.

The results include many revisions from the same user in quick succession.

The Village Pump is a busy page where users make many small edits. Use the comment field to identify distinct discussions, or post-process the data to group by section (if the comment contains a section header pattern).

FAQ

Question	Answer
What exactly does the Wikimedia Commons Village Pump scraper extract?	It extracts the revision history of the Commons Village Pump page (and its archives if specified) - each revision shows the user, timestamp, edit comment (which usually contains the section title and brief summary), and size change.
Does it require a Wikimedia API key or login?	No. The MediaWiki API is public and requires no authentication. This Actor uses it directly.
Can I filter only discussions about a specific topic?	Yes. Use the 'searchTerm' input. Only revisions whose comment (which includes the section title) contains that term will be returned.
How many revisions can I collect?	Free users get a preview of up to 10 items. Paid users can set maxItems up to 1,000,000.
Does it scrape the Village Pump archives too?	By default it scrapes the main Village Pump page. To target archives, you would need to add those specific pages manually - this is not yet built in automatically.
What export formats are supported?	CSV, JSON, Excel, and XML.
Is the data returned in real time?	It is fetched when you run the Actor, so it reflects the most recent available revisions up to that moment.
What fields are in each row?	Each row includes the revision ID, user, timestamp, comment, size, and page info. The exact fields appear in the sample record shown after a run.
Does this work for other Wikimedia Foundation wikis (Wikipedia, Wikidata)?	No - it is specifically built for the Commons Village Pump. Other wikis would need a different configuration.
Can I get the full discussion text (revision content) rather than the comment?	No. This scraper collects revision metadata only (user, timestamp, comment, size). To get the full page content at each revision, you would need a separate Actor.

Related actors

Browse the full ParseForge collection for more scrapers.

🆘 Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by Wikimedia Foundation. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.

💰 How much does it cost to scrape Wikimedia Commons Village Pump?

This Actor uses pay-per-result pricing: $0.004 per result collected. You are billed only for the results you receive, so a run that returns nothing costs nothing.