Wikimedia Commons Subcategory Scraper

Wikimedia Commons Subcategory Scraper

Scrape Wikimedia Commons subcategory trees from any category, up to a million per run. Every subcategory comes with its page ID, namespace, and full title. No login or API key. Export to CSV, JSON, Excel, or XML.

Wikimedia Commons holds millions of freely licensed media files, but they are organized in a deep hierarchy of categories. Finding every subcategory under a topic by hand is slow and error-prone. This scraper reads the public Wikimedia API directly, starting from one category and walking down the tree, so you get a complete list of subcategories in one dataset.

Who uses it	What they scrape Wikimedia Commons for
Digital archivists	Mapping the full category structure of a Commons topic for a GLAM project.
Data curators	Building a taxonomy of freely licensed media categories for a search tool.
Researchers	Analyzing how a subject area is categorized and organized on Commons.
Content managers	Finding all relevant subcategories to source images for a website or publication.

What it does

This Actor collects Wikimedia Commons subcategory names and IDs from a starting category and returns each one as a flat row.

• 🌳 Full tree traversal: starts at one category and recursively fetches all descendant subcategories.
• ⚙️ Configurable depth: set a maximum number of items to control the size of your dataset.
• 📄 Flat row output: each subcategory is one row with its page ID, namespace, and title, ready for spreadsheets.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

What you can do with Wikimedia Commons data

🗂️ Map a media category tree.

A digital archivist starts from 'Category:Paintings' and collects every subcategory to understand how the collection is organized before a migration.

🔍 Build a searchable taxonomy.

A developer scrapes all subcategories under 'Category:Animals' and uses the list to power a faceted search on a stock media website.

📊 Analyze category growth.

A researcher runs the scraper monthly on 'Category:COVID-19' and tracks how the number of subcategories changes over time.

🖼️ Source images by topic.

A content manager extracts all subcategories of 'Category:Historical photographs of cities' to find relevant image sets for a blog series.

Why choose this scraper

	What you get
No API key needed	Uses the public Wikimedia API with no registration or authentication.
Recursive discovery	Automatically walks down the category tree so you do not miss nested subcategories.
Fixed schema	Every row has the same fields: page ID, namespace, and full category title.
Flexible export	Save results as CSV, JSON, Excel, or XML for any downstream tool.

What a Wikimedia Commons record looks like

Every record returns as one flat JSON row. Here is a real one from a run:

{
 "pageid": 703561,
 "ns": 14,
 "title": "Category:Wikimedia Commons",
 "url": "https://commons.wikimedia.org/wiki/Category%3AWikimedia_Commons",
 "scrapedAt": "2026-09-04T01:16:46.381Z"
}

Every value above comes from a real run. A field a record does not have comes back as null.

Configure the run

Drive the Actor from a single starting category title, and set a maximum item count to limit how many subcategories are collected. The Input tab lists every parameter.

A first run with the defaults:

{
 "startCategory": "Category:Commons",
 "maxItems": 10
}

A larger pull:

{
 "startCategory": "Category:Commons",
 "maxItems": 200
}

Free users

Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.

Run it

1. Create a free Apify account with $5 in credit.
2. Open the Wikimedia Commons Subcategory Scraper.
3. Set your inputs and any filters, then click Start.
4. Export the results as CSV, Excel, JSON, or XML from the Dataset tab.

Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.

Use with AI agents (MCP)

Give an AI agent live access to Wikimedia Commons through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

$claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/wikimedia-commons-subcategory-scraper"

Then prompt it in plain language to run the scraper and read back the results.

Troubleshooting

Why am I getting no results?

Check that your start category title is spelled exactly as it appears on Commons, including the 'Category:' prefix and any spaces or punctuation. Also verify the category contains subcategories.

The run stopped before collecting all subcategories.

You likely hit the Max Items limit you set. Increase the number in the input to allow more subcategories to be collected.

I see duplicate subcategories in my dataset.

A subcategory can appear under multiple parent categories. The scraper returns every occurrence it finds during traversal, so duplicates are expected and reflect the real Commons structure.

The scraper is taking a long time to finish.

Large category trees with many thousands of subcategories take time because the API is queried page by page. Reduce the Max Items limit to get a smaller, faster sample.

FAQ

Question	Answer
What is a Wikimedia Commons subcategory?	A subcategory is a child category nested inside a parent category. For example, 'Category:Dogs' is a subcategory of 'Category:Animals'. This scraper collects the titles and IDs of those child categories.
Do I need a Wikimedia account or API key?	No. The scraper uses the public Wikimedia API, which requires no login, no token, and no registration.
How deep does the scraper go into the category tree?	It starts at the category you provide and recursively fetches all descendant subcategories until it reaches the maximum item count you set.
Can I scrape media files, not subcategories?	No, this Actor is designed to collect subcategory metadata only. To scrape actual media file URLs or metadata, you would need a different scraper.
What does the output look like?	Each row is a flat object with the subcategory's page ID, namespace (always 14 for categories), and full title including the 'Category:' prefix.
Is there a limit on how many subcategories I can scrape?	Free users are limited to 10 items for a preview. Paid users can set a maximum up to 1,000,000 items per run.
Can I start from any category?	Yes, you provide the full title with the 'Category:' prefix, such as 'Category:Commons' or 'Category:Paintings from Italy'.
What export formats are supported?	You can export your dataset to CSV, JSON, Excel, or XML from the Apify platform.

Related actors

Browse the full ParseForge collection for more scrapers.

🆘 Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.

Pricing

This Actor uses pay-per-result pricing: $0.004 per result collected. You are billed only for the results you receive, so a run that returns nothing costs nothing.

⚠️ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by Wikimedia Foundation, Inc. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.