WikiCFP Academic Conference CFP Scraper

🎓 Academic Conference CFP Scraper is a powerful Apify Actor designed to discover, track, and collect Academic Conference CFP (Call for Papers) listings directly from WikiCFP by keyword search. This tool provides comprehensive Academic Conference CFP intelligence including conference title, submission deadline, and direct listing link for any research topic or field. Whether you're building a research dashboard, tracking submission deadlines, or conducting academic outreach, the WikiCFP Academic Conference CFP Scraper delivers actionable Academic Conference CFP insights efficiently.

With direct WikiCFP integration, multi-keyword batch search, multi-page pagination, and reliable Apify Dataset delivery, the WikiCFP Academic Conference CFP Scraper ensures comprehensive Academic Conference CFP coverage across any research area. It focuses on key Academic Conference CFP signals including title, deadline, and listing link, making it an essential tool for academic research and conference-tracking workflows.

---

📋 Table of Contents

• Features
• Data Source
• How It Works
• Input
• Output
• Technical Stack
• Data Fields
• Use Cases
• Quick Start
• Configuration
• Performance
• Important Notes
• License & Legal

---

🔥 Features

• WikiCFP Search Integration – Direct integration with WikiCFP's public call-for-papers search interface.
• Multi-Keyword Batch Search – Search for Academic Conference CFP listings across any number of topics in a single run.
• Multi-Page Pagination – Automatically walks through multiple result pages per keyword to maximize Academic Conference CFP coverage.
• Deadline Extraction with Fallback – Pulls submission deadlines from the listing row, checking the following row if the primary cell is empty or non-textual.
• Global Deduplication – Tracks seen CFP IDs across all keywords and pages to avoid duplicate Academic Conference CFP records.
• Incremental Dataset Push – Pushes each page's results to Apify Dataset immediately, rather than waiting for the full run to finish.
• Automatic Pagination Stop – Moves to the next keyword automatically once a page returns no further Academic Conference CFP listings.
• Rate-Limit & Retry Handling – Automatically retries on 401/403/429 responses with exponential backoff.
• Residential Proxy Support – Apify residential proxy for reliable WikiCFP access.
• Detailed Logging – Comprehensive per-keyword and per-page Academic Conference CFP progress logging.
• Graceful Error Handling – Skips a failing page or keyword without stopping the whole run.

---

📊 Data Source

WikiCFP Public Call for Papers Directory

• Authority: WikiCFP, a long-running community-maintained directory of academic calls for papers
• Endpoint: wikicfp.com/cfp/call
• Coverage: Academic conferences and workshops across virtually every research field
• Data: Conference title, submission deadline, and direct event link
• Access: Public, key-less HTML search interface
• Update Frequency: Reflects WikiCFP's current listings at time of run

---

⚙️ How It Works

The WikiCFP Academic Conference CFP Scraper accepts one or more search keywords and queries WikiCFP's public call-for-papers search for each one, paginating through results until either the requested page count is reached or a page returns no further listings. Each result row is parsed to extract the CFP ID, title, and submission deadline — checking the following table row as a fallback when the deadline cell is missing or non-textual. Results are deduplicated globally by CFP ID across all keywords and pages, and each page's new Academic Conference CFP records are pushed to the Apify Dataset immediately.

Key Processing Steps:

1. Input Parsing – Accept one or more search keywords and a page count per keyword
2. Proxy Setup – Configure Apify residential proxy
3. Session Init – Create a Chrome-impersonated HTTP session for reliable WikiCFP access
4. Keyword Loop – Iterate through each requested Academic Conference CFP search term
5. Page Loop – Request each page of results up to the configured page count
6. HTML Parsing – Extract CFP ID, title, link, and deadline from each listing row
7. Deadline Fallback – Check the next table row if the primary deadline cell is empty
8. Deduplication – Skip any CFP ID already seen in this run
9. Incremental Push – Push each page's new Academic Conference CFP records immediately
10. Auto-Stop – Move to the next keyword once a page returns no new listings
11. Rate Limiting – Apply a short delay between page requests
12. Summary Logging – Report the total number of CFPs extracted at the end of the run

Key Benefits:

• Pull Academic Conference CFP listings for any research topic without manual browsing
• Track submission deadlines across multiple fields in one consolidated dataset
• Build a recurring watchlist of conferences relevant to your research area
• Feed academic research pipelines, deadline trackers, or outreach tools
• Automate recurring Academic Conference CFP checks as WikiCFP adds new listings

---

📥 Input

The Actor accepts the following input parameters:

Field	Type	Default	Description
keywords	array	(optional)	List of search terms for Academic Conference CFP listings.
keyword	string	"machine learning"	Single-keyword fallback used if keywords is not provided.
pages	integer	2	Number of result pages to fetch per keyword.
proxyConfiguration	object	{"useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"]}	Apify Proxy configuration.

Example Input:

{
  "keywords": ["machine learning", "natural language processing"],
  "pages": 3,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}

Search a Single Topic:

{
  "keyword": "computer vision"
}

Search Multiple Topics with More Pages:

{
  "keywords": ["robotics", "cybersecurity", "bioinformatics"],
  "pages": 5
}

---

📤 Output

The Actor pushes Academic Conference CFP records with the following structure:

Field	Type	Description
keyword	string	The search term that produced this Academic Conference CFP listing
cfp_id	string	WikiCFP's unique event ID
title	string	Conference or workshop title
deadline	string	Submission deadline text as listed on WikiCFP
link	string	Direct URL to the full Academic Conference CFP listing page
scraped_at	string	ISO 8601 timestamp of when the record was scraped

Example Academic Conference CFP Record:

{
  "keyword": "machine learning",
  "cfp_id": "123456",
  "title": "International Conference on Machine Learning and Applications",
  "deadline": "Sep 15, 2026",
  "link": "http://www.wikicfp.com/cfp/servlet/event.showcfp?eventid=123456",
  "scraped_at": "2026-08-17T09:00:00+00:00"
}

---

🧰 Technical Stack

• HTTP Client: `` with Chrome TLS fingerprint impersonation for reliable Academic Conference CFP page access
• HTML Parsing: BeautifulSoup for listing row and deadline extraction
• Date Handling: datetime with timezone-aware UTC timestamps
• Async: asyncio with executor-run blocking requests for non-blocking, multi-keyword processing
• Proxy: Apify Proxy with RESIDENTIAL configuration
• Logging: Apify Actor logging system
• Platform: Apify Actor serverless environment

---

📊 Data Fields Explained

Identification

• cfp_id / link: The unique WikiCFP event ID and direct link to the full Academic Conference CFP listing
• keyword: The search term under which this listing was discovered

Listing Details

• title: The name of the conference or workshop
• deadline: The submission deadline as published on WikiCFP

Timing

• scraped_at: When this Academic Conference CFP record was collected

---

🎯 Use Cases

• Deadline Tracking – Stay on top of submission deadlines across multiple research areas
• Academic Outreach – Identify relevant conferences for paper submissions or collaboration
• Research Planning – Build a calendar of upcoming Academic Conference CFP opportunities
• Grant & Funding Research – Cross-reference conference activity with funding cycles
• Institutional Reporting – Track conference participation opportunities for a department or lab
• Academic Newsletter Curation – Source fresh CFP listings for a research community newsletter
• Trend Analysis – Monitor how many CFPs appear for a given topic over time

---

🚀 Quick Start

1. Prepare Input

Go to Apify Console and enter:

{
  "keywords": ["machine learning", "computer vision"],
  "pages": 2
}

2. Run the Actor

Click Start. The Actor will:

• Search WikiCFP for each requested keyword
• Paginate through results until listings run out or the page limit is hit
• Deduplicate CFPs across keywords and pages
• Push new Academic Conference CFP records to the Dataset as they're found

3. Monitor Progress

Console shows:

[1/2] Searching WikiCFP for: 'machine learning' (2 pages)...
Saved 15 CFPs from page 1 for 'machine learning'
Saved 12 CFPs from page 2 for 'machine learning'
[2/2] Searching WikiCFP for: 'computer vision' (2 pages)...
Saved 18 CFPs from page 1 for 'computer vision'
🎉 Scraping complete! Successfully extracted 45 CFPs.

4. View & Download Results

• Results Tab: All Academic Conference CFP records
• Export: JSON, CSV, Excel, or HTML
• Filter: By keyword, deadline, or title
• API Access: Available via the Apify API

---

⚙️ Configuration

Single keyword, default pages:

{
  "keyword": "data mining"
}

Multiple keywords with more pages:

{
  "keywords": ["artificial intelligence", "quantum computing"],
  "pages": 4
}

Run without proxy:

{
  "keyword": "bioinformatics",
  "proxyConfiguration": {"useApifyProxy": false}
}

---

📈 Performance

Processing Speed

• One request per page per keyword, with automatic retry on rate limits
• 0.5 second delay between page requests to reduce blocking
• Stops early per keyword once a page returns no listings, avoiding wasted requests

Resource Usage

• Memory: Low, since no browser instance is launched
• Network: One request per result page across all requested keywords
• Proxy: One residential proxy tunnel shared across the session for the run

---

⚠️ Important Notes

Legal & Compliance

• Fair Use: Respects WikiCFP's public listing pages and applies pacing delays between requests
• Public Data Only: Retrieves only publicly listed Academic Conference CFP information
• Legal: Not legal advice — consult qualified professionals before using this data for compliance-sensitive decisions

Data Quality

• Freshness: Reflects WikiCFP's listings at time of run
• Completeness: Deadline text depends on how each conference organizer formatted their listing; some entries may have incomplete deadline info
• Accuracy: Sourced directly from WikiCFP's public directory
• Verification: Cross-check high-stakes deadlines against the official conference website before submitting

Best Practices

• Use specific, well-known field terms (e.g., "natural language processing" rather than "NLP") for the most complete WikiCFP matches
• Increase pages gradually to balance coverage against run time
• Combine related keywords in one run to build a broader Academic Conference CFP dataset
• Re-run periodically to catch newly posted CFPs
• Always verify deadlines directly on the official conference page before final submission

---

📄 License & Legal

Terms of Use:

• Use for legitimate academic research, outreach, and planning purposes
• Respect WikiCFP's Terms of Service
• Attribute WikiCFP as the original data source where required
• Use data responsibly

Disclaimer: WikiCFP Academic Conference CFP Scraper is provided as-is for research purposes. Users are responsible for compliance with WikiCFP's ToS. This is not legal advice, and deadlines should always be verified against the official conference source.

---

⚖️ Academic Conference CFP Excellence

This Actor is optimized for Academic Conference CFP research with:

• ✅ Multi-keyword, multi-page batch search
• ✅ Global deduplication across the full run
• ✅ Resilient deadline extraction with row fallback
• ✅ Incremental, real-time Dataset push
• ✅ Rate-limit-aware retry logic
• ✅ Production-ready code