Wikipedia Scraper Actor

📖 What does it do?

The Wikipedia Scraper is your one-stop shop for mining Wikipedia like a pro. It extracts article titles, URLs, and last modification dates from English (EN) or German (DE) Wikipedia. Whether you need just a handful of results or an exhaustive list, this scraper has you covered—no fluff, just actionable data.

---

🚀 Why use this Actor?

• Exact Phrase Searches: The scraper runs precise searches by wrapping your search term in quotes. No more messy, irrelevant results!
• Language Selection: Choose between EN and DE Wikipedia for multilingual data mining.
• Pagination Power: This Actor handles pagination like a champ—extract up to 1000 results in one run.
• Developer Friendly: Flexible input schema, detailed logs, and outputs in a structured JSON/CSV/XML/EXCEL/HTML/RSS/JSONL format.

---

🛠️ How does it work?

1. Input your parameters:

   • language: Choose between EN Wikipedia or DE Wikipedia.
   • search_term: Provide your search term. Make it specific for the best results.
   • max_results: Decide how many results you want (default: 50, max: 1000).

2. Hit the Run Button:
   The Actor navigates Wikipedia search pages, extracts articles, handles pagination, and pushes all the good stuff into the output.

3. Get your data:
   Titles, URLs, and last modification dates in a clean, developer-ready format.

---

📝 Input Parameters

Parameter	Type	Default	Description
language	String	EN Wikipedia	Choose between EN Wikipedia or DE Wikipedia.
search_term	String	Required	The exact phrase you want to search for.
max_results	Integer	50	Number of results to scrape (min: 1, max: 1000).

---

🛑 Caveats

• If your search term is too generic, Wikipedia might throw irrelevant results your way. Be precise!
• The Actor assumes your search term is an exact match (wrapped in quotes). No approximate matching here—precision is key.
• Results exceeding max_results are ignored. Configure appropriately.

---

📦 Output

You’ll get structured data, containing:

• language: The Wikipedia language version used.
• title: The cleaned article title (no "– Wikipedia" nonsense).
• url: The article URL.
• last_modified: The raw last-modified string.
• last_modified_date: The last-modified date in Day Month Year format.

Example:

1{
2  "search_term": "Stiftung Schweiz",
3  "language": "DE Wikipedia",
4  "title": "Stiftung (Schweiz)",
5  "url": "https://de.wikipedia.org/wiki/Stiftung_(Schweiz)",
6  "last_modified": "Diese Seite wurde zuletzt am 18. Juni 2024 um 08:44 Uhr bearbeitet.",
7  "last_modified_date": "18. Juni 2024"
8}

---

🔧 Under the Hood

This Actor uses Playwright for precise scraping and Apify's Request Queue for managing pagination. It’s all written in Python, so feel free to fork and extend it to suit your needs.

---

🤔 Need Help?

If you have questions, suggestions, or just want to share how this Actor made your life easier, feel free to reach out on the Apify community forums.

---

🎉 Test it Today! Stop manually clicking through Wikipedia pages and let this Actor do the heavy lifting. Search smarter, scrape faster, and enjoy the magic of automation.

🎉 Test it Today!

Stop manually clicking through Wikipedia pages and let this Actor do the heavy lifting. Search smarter, scrape faster, and enjoy the magic of automation—try it now!