# WikiCFP Academic Conference CFP Scraper (`datapilot/wikicfp-academic-conference-cfp-scraper`) Actor

Extract CFP ID, conference title, submission deadline, direct CFP link, and scrape timestamp. Supports multiple pages, keyword searches, duplicate removal, Residential Proxy, retries, and structured Apify Dataset output.

- **URL**: https://apify.com/datapilot/wikicfp-academic-conference-cfp-scraper.md
- **Developed by:** [Data Pilot](https://apify.com/datapilot) (community)
- **Categories:** Other
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.00 / 1,000 result scrapes

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

🎓 **Academic Conference CFP** Scraper is a powerful Apify Actor designed to discover, track, and collect **Academic Conference CFP** (Call for Papers) listings directly from WikiCFP by keyword search. This tool provides comprehensive **Academic Conference CFP** intelligence including conference title, submission deadline, and direct listing link for any research topic or field. Whether you're building a research dashboard, tracking submission deadlines, or conducting academic outreach, the WikiCFP Academic Conference CFP Scraper delivers actionable **Academic Conference CFP** insights efficiently.

With direct WikiCFP integration, multi-keyword batch search, multi-page pagination, and reliable Apify Dataset delivery, the WikiCFP Academic Conference CFP Scraper ensures comprehensive **Academic Conference CFP** coverage across any research area. It focuses on key **Academic Conference CFP** signals including title, deadline, and listing link, making it an essential tool for academic research and conference-tracking workflows.

***

### 📋 Table of Contents

- [Features](#-features)
- [Data Source](#-data-source)
- [How It Works](#-how-it-works)
- [Input](#-input)
- [Output](#-output)
- [Technical Stack](#-technical-stack)
- [Data Fields](#-data-fields-explained)
- [Use Cases](#-use-cases)
- [Quick Start](#-quick-start)
- [Configuration](#-configuration)
- [Performance](#-performance)
- [Important Notes](#-important-notes)
- [License & Legal](#-license--legal)

***

### 🔥 Features

- **WikiCFP Search Integration** – Direct integration with WikiCFP's public call-for-papers search interface.
- **Multi-Keyword Batch Search** – Search for **Academic Conference CFP** listings across any number of topics in a single run.
- **Multi-Page Pagination** – Automatically walks through multiple result pages per keyword to maximize **Academic Conference CFP** coverage.
- **Deadline Extraction with Fallback** – Pulls submission deadlines from the listing row, checking the following row if the primary cell is empty or non-textual.
- **Global Deduplication** – Tracks seen CFP IDs across all keywords and pages to avoid duplicate **Academic Conference CFP** records.
- **Incremental Dataset Push** – Pushes each page's results to Apify Dataset immediately, rather than waiting for the full run to finish.
- **Automatic Pagination Stop** – Moves to the next keyword automatically once a page returns no further **Academic Conference CFP** listings.
- **Rate-Limit & Retry Handling** – Automatically retries on 401/403/429 responses with exponential backoff.
- **Residential Proxy Support** – Apify residential proxy for reliable WikiCFP access.
- **Detailed Logging** – Comprehensive per-keyword and per-page **Academic Conference CFP** progress logging.
- **Graceful Error Handling** – Skips a failing page or keyword without stopping the whole run.

***

### 📊 Data Source

#### **WikiCFP Public Call for Papers Directory**

- **Authority**: WikiCFP, a long-running community-maintained directory of academic calls for papers
- **Endpoint**: `wikicfp.com/cfp/call`
- **Coverage**: Academic conferences and workshops across virtually every research field
- **Data**: Conference title, submission deadline, and direct event link
- **Access**: Public, key-less HTML search interface
- **Update Frequency**: Reflects WikiCFP's current listings at time of run

***

### ⚙️ How It Works

The WikiCFP Academic Conference CFP Scraper accepts one or more search keywords and queries WikiCFP's public call-for-papers search for each one, paginating through results until either the requested page count is reached or a page returns no further listings. Each result row is parsed to extract the CFP ID, title, and submission deadline — checking the following table row as a fallback when the deadline cell is missing or non-textual. Results are deduplicated globally by CFP ID across all keywords and pages, and each page's new **Academic Conference CFP** records are pushed to the Apify Dataset immediately.

**Key Processing Steps:**

1. **Input Parsing** – Accept one or more search keywords and a page count per keyword
2. **Proxy Setup** – Configure Apify residential proxy
3. **Session Init** – Create a Chrome-impersonated HTTP session for reliable WikiCFP access
4. **Keyword Loop** – Iterate through each requested **Academic Conference CFP** search term
5. **Page Loop** – Request each page of results up to the configured page count
6. **HTML Parsing** – Extract CFP ID, title, link, and deadline from each listing row
7. **Deadline Fallback** – Check the next table row if the primary deadline cell is empty
8. **Deduplication** – Skip any CFP ID already seen in this run
9. **Incremental Push** – Push each page's new **Academic Conference CFP** records immediately
10. **Auto-Stop** – Move to the next keyword once a page returns no new listings
11. **Rate Limiting** – Apply a short delay between page requests
12. **Summary Logging** – Report the total number of CFPs extracted at the end of the run

**Key Benefits:**

- Pull **Academic Conference CFP** listings for any research topic without manual browsing
- Track submission deadlines across multiple fields in one consolidated dataset
- Build a recurring watchlist of conferences relevant to your research area
- Feed academic research pipelines, deadline trackers, or outreach tools
- Automate recurring **Academic Conference CFP** checks as WikiCFP adds new listings

***

### 📥 Input

The Actor accepts the following input parameters:

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `keywords` | array | *(optional)* | List of search terms for **Academic Conference CFP** listings. |
| `keyword` | string | `"machine learning"` | Single-keyword fallback used if `keywords` is not provided. |
| `pages` | integer | `2` | Number of result pages to fetch per keyword. |
| `proxyConfiguration` | object | `{"useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"]}` | Apify Proxy configuration. |

**Example Input:**

```json
{
  "keywords": ["machine learning", "natural language processing"],
  "pages": 3,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}
```

**Search a Single Topic:**

```json
{
  "keyword": "computer vision"
}
```

**Search Multiple Topics with More Pages:**

```json
{
  "keywords": ["robotics", "cybersecurity", "bioinformatics"],
  "pages": 5
}
```

***

### 📤 Output

The Actor pushes **Academic Conference CFP** records with the following structure:

| Field | Type | Description |
|-------|------|-------------|
| `keyword` | string | The search term that produced this **Academic Conference CFP** listing |
| `cfp_id` | string | WikiCFP's unique event ID |
| `title` | string | Conference or workshop title |
| `deadline` | string | Submission deadline text as listed on WikiCFP |
| `link` | string | Direct URL to the full **Academic Conference CFP** listing page |
| `scraped_at` | string | ISO 8601 timestamp of when the record was scraped |

**Example Academic Conference CFP Record:**

```json
{
  "keyword": "machine learning",
  "cfp_id": "123456",
  "title": "International Conference on Machine Learning and Applications",
  "deadline": "Sep 15, 2026",
  "link": "http://www.wikicfp.com/cfp/servlet/event.showcfp?eventid=123456",
  "scraped_at": "2026-08-17T09:00:00+00:00"
}
```

***

### 🧰 Technical Stack

- **HTTP Client:** \`\` with Chrome TLS fingerprint impersonation for reliable **Academic Conference CFP** page access
- **HTML Parsing:** BeautifulSoup for listing row and deadline extraction
- **Date Handling:** `datetime` with timezone-aware UTC timestamps
- **Async:** `asyncio` with executor-run blocking requests for non-blocking, multi-keyword processing
- **Proxy:** Apify Proxy with `RESIDENTIAL` configuration
- **Logging:** Apify Actor logging system
- **Platform:** Apify Actor serverless environment

***

### 📊 Data Fields Explained

#### **Identification**

- **cfp\_id / link**: The unique WikiCFP event ID and direct link to the full **Academic Conference CFP** listing
- **keyword**: The search term under which this listing was discovered

#### **Listing Details**

- **title**: The name of the conference or workshop
- **deadline**: The submission deadline as published on WikiCFP

#### **Timing**

- **scraped\_at**: When this **Academic Conference CFP** record was collected

***

### 🎯 Use Cases

- **Deadline Tracking** – Stay on top of submission deadlines across multiple research areas
- **Academic Outreach** – Identify relevant conferences for paper submissions or collaboration
- **Research Planning** – Build a calendar of upcoming **Academic Conference CFP** opportunities
- **Grant & Funding Research** – Cross-reference conference activity with funding cycles
- **Institutional Reporting** – Track conference participation opportunities for a department or lab
- **Academic Newsletter Curation** – Source fresh CFP listings for a research community newsletter
- **Trend Analysis** – Monitor how many CFPs appear for a given topic over time

***

### 🚀 Quick Start

#### **1. Prepare Input**

Go to Apify Console and enter:

```json
{
  "keywords": ["machine learning", "computer vision"],
  "pages": 2
}
```

#### **2. Run the Actor**

Click **Start**. The Actor will:

- Search WikiCFP for each requested keyword
- Paginate through results until listings run out or the page limit is hit
- Deduplicate CFPs across keywords and pages
- Push new **Academic Conference CFP** records to the Dataset as they're found

#### **3. Monitor Progress**

Console shows:

```
[1/2] Searching WikiCFP for: 'machine learning' (2 pages)...
Saved 15 CFPs from page 1 for 'machine learning'
Saved 12 CFPs from page 2 for 'machine learning'
[2/2] Searching WikiCFP for: 'computer vision' (2 pages)...
Saved 18 CFPs from page 1 for 'computer vision'
🎉 Scraping complete! Successfully extracted 45 CFPs.
```

#### **4. View & Download Results**

- **Results Tab**: All **Academic Conference CFP** records
- **Export**: JSON, CSV, Excel, or HTML
- **Filter**: By keyword, deadline, or title
- **API Access**: Available via the Apify API

***

### ⚙️ Configuration

Single keyword, default pages:

```json
{
  "keyword": "data mining"
}
```

Multiple keywords with more pages:

```json
{
  "keywords": ["artificial intelligence", "quantum computing"],
  "pages": 4
}
```

Run without proxy:

```json
{
  "keyword": "bioinformatics",
  "proxyConfiguration": {"useApifyProxy": false}
}
```

***

### 📈 Performance

#### **Processing Speed**

- One request per page per keyword, with automatic retry on rate limits
- 0.5 second delay between page requests to reduce blocking
- Stops early per keyword once a page returns no listings, avoiding wasted requests

#### **Resource Usage**

- Memory: Low, since no browser instance is launched
- Network: One request per result page across all requested keywords
- Proxy: One residential proxy tunnel shared across the session for the run

***

### ⚠️ Important Notes

#### **Legal & Compliance**

- **Fair Use**: Respects WikiCFP's public listing pages and applies pacing delays between requests
- **Public Data Only**: Retrieves only publicly listed **Academic Conference CFP** information
- **Legal**: Not legal advice — consult qualified professionals before using this data for compliance-sensitive decisions

#### **Data Quality**

- **Freshness**: Reflects WikiCFP's listings at time of run
- **Completeness**: Deadline text depends on how each conference organizer formatted their listing; some entries may have incomplete deadline info
- **Accuracy**: Sourced directly from WikiCFP's public directory
- **Verification**: Cross-check high-stakes deadlines against the official conference website before submitting

#### **Best Practices**

- Use specific, well-known field terms (e.g., "natural language processing" rather than "NLP") for the most complete WikiCFP matches
- Increase `pages` gradually to balance coverage against run time
- Combine related keywords in one run to build a broader **Academic Conference CFP** dataset
- Re-run periodically to catch newly posted CFPs
- Always verify deadlines directly on the official conference page before final submission

***

### 📄 License & Legal

**Terms of Use:**

- Use for legitimate academic research, outreach, and planning purposes
- Respect WikiCFP's Terms of Service
- Attribute WikiCFP as the original data source where required
- Use data responsibly

**Disclaimer:**
WikiCFP Academic Conference CFP Scraper is provided as-is for research purposes. Users are responsible for compliance with WikiCFP's ToS. This is not legal advice, and deadlines should always be verified against the official conference source.

***

### ⚖️ Academic Conference CFP Excellence

This Actor is optimized for **Academic Conference CFP** research with:

- ✅ Multi-keyword, multi-page batch search
- ✅ Global deduplication across the full run
- ✅ Resilient deadline extraction with row fallback
- ✅ Incremental, real-time Dataset push
- ✅ Rate-limit-aware retry logic
- ✅ Production-ready code

# Actor input Schema

## `keywords` (type: `array`):

List of conference names, topics, or research fields (e.g. 'machine learning', 'cybersecurity').

## `pages` (type: `integer`):

Number of result pages to crawl per keyword (around 20 CFPs per page).

## `proxyConfiguration` (type: `object`):

Select Residential Proxy to bypass blocks and avoid rate limiting.

## Actor input object example

```json
{
  "keywords": [
    "machine learning",
    "data science"
  ],
  "pages": 2,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}
```

# Actor output Schema

## `crawlResults` (type: `string`):

No description

## `cfpsOverview` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "keywords": [
        "machine learning",
        "data science"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("datapilot/wikicfp-academic-conference-cfp-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "keywords": [
        "machine learning",
        "data science",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("datapilot/wikicfp-academic-conference-cfp-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "keywords": [
    "machine learning",
    "data science"
  ]
}' |
apify call datapilot/wikicfp-academic-conference-cfp-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,datapilot/wikicfp-academic-conference-cfp-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/89gYGrHAbys6nDH5z/builds/E4hxO5qBKQP0Gpzcv/openapi.json