# Researchgpt Deep Research Agent (`wheat_tourist/researchgpt-deep-research-agent`) Actor

🔬 Transform any topic into a comprehensive research report in minutes! Scrapes Wikipedia, arXiv, Semantic Scholar, news & web sources. Outputs professional JSON, HTML & PDF reports. Perfect for students, researchers, content creators & businesses. No API keys needed.

- **URL**: https://apify.com/wheat_tourist/researchgpt-deep-research-agent.md
- **Developed by:** [Varun Chopra](https://apify.com/wheat_tourist) (community)
- **Categories:** AI, Automation, News
- **Stats:** 91 total users, 2 monthly users, 89.3% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $35.00 / 1,000 results

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## 🔬 ResearchGPT - Deep Research Agent

#### *Transform any topic into a comprehensive research report in minutes, not hours.*

[![Apify Actor](https://img.shields.io/badge/Apify-Actor-blue?logo=apify)](https://apify.com)
[![Python 3.11+](https://img.shields.io/badge/Python-3.11+-green?logo=python)](https://python.org)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

***

### 🎯 What is ResearchGPT?

**ResearchGPT** is your AI-powered research assistant that does in **3 minutes** what would take you **3+ hours** manually.

Simply enter any topic, and ResearchGPT will:

✅ **Search** across multiple engines (DuckDuckGo, Brave, Mojeek)\
✅ **Scrape** Wikipedia, arXiv, Semantic Scholar, OpenAlex, CrossRef\
✅ **Extract** the latest news articles and web content\
✅ **Process** everything with intelligent NLP analysis\
✅ **Generate** beautiful reports in JSON, HTML & PDF formats

**No API keys required. No complex setup. Just results.**

***

### 🚀 Perfect For

| Use Case | How ResearchGPT Helps |
|----------|----------------------|
| 📚 **Students & Academics** | Literature reviews, thesis research, citation gathering |
| ✍️ **Content Creators** | Blog research, fact-checking, source compilation |
| 💼 **Business Analysts** | Market research, competitive analysis, trend reports |
| 🔬 **Researchers** | Cross-referencing sources, academic paper aggregation |
| 📰 **Journalists** | Background research, source verification, story development |
| 🤖 **AI/ML Projects** | Training data collection, knowledge base building |

***

### ⚡ Quick Start

#### 1. Run on Apify (Easiest)

1. Go to the [ResearchGPT Actor page](https://apify.com/your-username/researchgpt-deep-research-agent)
2. Enter your research topic
3. Click **Start**
4. Download your reports! 📄

#### 2. Via API

```bash
curl -X POST "https://api.apify.com/v2/acts/YOUR_USERNAME~researchgpt-deep-research-agent/runs?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"topic": "quantum computing breakthroughs 2025"}'
```

#### 3. Via Apify SDK (Python)

```python
from apify_client import ApifyClient

client = ApifyClient("YOUR_API_TOKEN")

run = client.actor("YOUR_USERNAME/researchgpt-deep-research-agent").call(
    run_input={"topic": "artificial intelligence in healthcare"}
)

## Get results
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)
```

***

### 📊 What You Get

#### Three Professional Output Formats

| Format | Best For | Contents |
|--------|----------|----------|
| 📄 **JSON** | Developers, APIs, databases | Full structured data with metadata |
| 🌐 **HTML** | Web publishing, sharing | Beautifully styled report with CSS |
| 📑 **PDF** | Printing, presentations | Clean, professional document |

#### Rich Research Data

```json
{
  "topic": "artificial intelligence ethics",
  "sources": {
    "wikipedia": 5,
    "academic": 10,
    "news": 5,
    "general": 10
  },
  "processed_content": {
    "summary": "Comprehensive executive summary...",
    "key_findings": ["Finding 1", "Finding 2", "..."],
    "themes": ["Theme 1", "Theme 2", "..."],
    "entities": ["Entity 1", "Entity 2", "..."]
  }
}
```

***

### 🔧 Configuration Options

```json
{
  "topic": "your research topic here",
  "outputFormats": ["json", "html", "pdf"],
  "maxSourcesPerType": 10,
  "includeWikipedia": true,
  "includeAcademic": true,
  "includeNews": true,
  "includeGeneral": true,
  "searchProviders": ["duckduckgo"],
  "requestTimeout": 30,
  "maxRetries": 3,
  "debug": false
}
```

#### Parameter Reference

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `topic` | string | *required* | 🎯 Your research topic or question |
| `outputFormats` | array | `["json", "html", "pdf"]` | 📄 Output formats to generate |
| `maxSourcesPerType` | integer | `10` | 📊 Sources per category (1-20) |
| `includeWikipedia` | boolean | `true` | 📖 Include Wikipedia articles |
| `includeAcademic` | boolean | `true` | 🎓 Include academic papers |
| `includeNews` | boolean | `true` | 📰 Include news articles |
| `includeGeneral` | boolean | `true` | 🌐 Include general web content |
| `searchProviders` | array | `["duckduckgo"]` | 🔍 Search engines to use |
| `requestTimeout` | integer | `30` | ⏱️ Request timeout (seconds) |
| `maxRetries` | integer | `3` | 🔄 Retry attempts on failure |
| `proxyConfiguration` | object | `null` | 🛡️ Apify proxy settings |
| `debug` | boolean | `false` | 🐛 Enable verbose logging |

***

### 🌐 Data Sources

ResearchGPT taps into **6+ authoritative sources**:

| Source | Type | What You Get |
|--------|------|--------------|
| 🌍 **Wikipedia** | Knowledge Base | Foundational articles via MediaWiki API |
| 📚 **arXiv** | Academic | Pre-print papers in physics, CS, math, and more |
| 🔬 **Semantic Scholar** | Academic | 200M+ papers with citation analysis |
| 📖 **OpenAlex** | Academic | Open catalog of scholarly works |
| 📑 **CrossRef** | Academic | DOI metadata and citations |
| 📰 **News Sources** | Current Events | Latest articles via smart extraction |
| 🌐 **General Web** | Insights | Curated web content with readability algorithms |

***

### 🛡️ Production-Grade Features

Built for reliability and scale:

- ⚡ **Smart Caching** - 5-minute TTL prevents redundant requests
- 🔄 **Retry Logic** - Exponential backoff with jitter
- 🚦 **Rate Limiting** - Respects API limits automatically
- 🔍 **Deduplication** - MD5-based content fingerprinting
- 🌐 **Connection Pooling** - Efficient HTTP management
- 🛡️ **Error Handling** - Graceful fallbacks, never crashes

***

### 💻 Local Development

```bash
## Clone the repository
git clone https://github.com/your-repo/researchgpt-deep-research-agent
cd researchgpt-deep-research-agent

## Create virtual environment
python -m venv .venv
.venv\Scripts\activate      # Windows
source .venv/bin/activate   # macOS/Linux

## Install dependencies
pip install -r requirements.txt

## Run locally
python run_local.py
```

***

### 📁 Project Structure

```
researchgpt-deep-research-agent/
├── 📂 .actor/
│   └── actor.json           # Apify configuration
├── 📂 src/
│   ├── __init__.py
│   └── main.py              # 🚀 Main Apify entry point
├── 📂 scrapers/
│   ├── base_scraper.py      # Base class with retry/caching
│   ├── academic_scraper.py  # arXiv, Semantic Scholar, etc.
│   ├── wikipedia_scraper.py # MediaWiki API
│   ├── news_scraper.py      # News extraction
│   ├── general_scraper.py   # Web scraping
│   └── search_engine.py     # Multi-provider search
├── 📂 processors/
│   └── content_processor.py # NLP processing
├── 📂 output/
│   └── output_generator.py  # Report generation
├── 📄 Dockerfile            # Container definition
├── 📄 requirements.txt      # Dependencies
└── 📄 README.md             # You are here!
```

***

### 🚀 Deploy to Apify

#### Option 1: Apify CLI (Recommended)

```bash
npm install -g apify-cli
apify login
apify push
```

#### Option 2: GitHub Integration

1. Push to GitHub
2. Apify Console → Create Actor → Link to GitHub
3. Auto-builds on every push! 🔄

***

### 📈 Performance Tips

| Tip | Impact |
|-----|--------|
| Lower `maxSourcesPerType` | ⚡ Faster results |
| Disable unused sources | 🚀 Skip what you don't need |
| Use single search provider | 📉 Reduce API calls |
| Enable `debug` mode | 🔍 Troubleshoot issues |

***

### 🤔 FAQ

<details>
<summary><strong>How long does it take?</strong></summary>

Typically 1-3 minutes depending on topic complexity and number of sources.

</details>

<details>
<summary><strong>Do I need API keys?</strong></summary>

No! ResearchGPT uses public APIs and smart scraping. No setup required.

</details>

<details>
<summary><strong>Can I use this for commercial projects?</strong></summary>

Yes! MIT licensed. Use it however you want.

</details>

<details>
<summary><strong>What if a source is blocked?</strong></summary>

ResearchGPT gracefully handles blocks and continues with other sources.

</details>

<details>
<summary><strong>Can I customize the output?</strong></summary>

Yes! Choose your formats (JSON/HTML/PDF) and configure source types.

</details>

***

### 🆚 Why ResearchGPT?

| Feature | ResearchGPT | Manual Research | Other Tools |
|---------|-------------|-----------------|-------------|
| ⏱️ **Time** | 3 minutes | 3+ hours | 30+ minutes |
| 📚 **Sources** | 6+ databases | Limited | Usually 1-2 |
| 📄 **Output** | JSON + HTML + PDF | Manual formatting | Single format |
| 💰 **Cost** | Pay per run | Your time = $$$$ | Subscription |
| 🔧 **Setup** | Zero | N/A | API keys needed |

***

### 📝 Example Topics

Get inspired! Here are some topics that work great:

- `"artificial intelligence ethics and regulation 2025"`
- `"quantum computing practical applications"`
- `"climate change solutions renewable energy"`
- `"cryptocurrency DeFi market analysis"`
- `"remote work productivity research"`
- `"mental health digital therapeutics"`
- `"gene editing CRISPR medical applications"`
- `"electric vehicles battery technology"`

***

### 🤝 Support & Community

- 🐛 **Issues**: [Report bugs](https://github.com/your-repo/issues)
- 💡 **Feature Requests**: [Suggest ideas](https://github.com/your-repo/issues)
- 📖 **Docs**: [Apify Documentation](https://docs.apify.com)
- 💬 **Discord**: [Join Apify Community](https://discord.gg/apify)

***

### 📄 License

MIT License - Use it freely, commercially or personally.

***

<div align="center">

#### ⭐ Star this repo if ResearchGPT saved you time!

**Built with ❤️ for researchers everywhere**

[Get Started](https://apify.com) • [Documentation](https://docs.apify.com) • [Support](https://github.com/your-repo/issues)

</div>

# Actor input Schema

## `topic` (type: `string`):

The topic you want to research. Be specific for better results.

## `outputFormats` (type: `array`):

Select the output formats you want to generate

## `maxSourcesPerType` (type: `integer`):

Maximum number of sources to scrape from each category (Wikipedia, Academic, News, General)

## `includeWikipedia` (type: `boolean`):

Scrape Wikipedia articles for foundational knowledge

## `includeAcademic` (type: `boolean`):

Scrape academic papers from arXiv, Semantic Scholar, OpenAlex, CrossRef

## `includeNews` (type: `boolean`):

Scrape recent news articles about the topic

## `includeGeneral` (type: `boolean`):

Scrape general web content for additional context

## `searchProviders` (type: `array`):

Search engines to use for URL discovery

## `requestTimeout` (type: `integer`):

HTTP request timeout in seconds

## `maxRetries` (type: `integer`):

Maximum retry attempts for failed requests

## `proxyConfiguration` (type: `object`):

Apify Proxy configuration (optional)

## `debug` (type: `boolean`):

Enable verbose logging for troubleshooting

## `enableSmartCitations` (type: `boolean`):

Analyze how sources relate to each other (supports, contrasts, or mentions)

## `generateInsights` (type: `boolean`):

Extract actionable insights with confidence ratings from research

## `identifyResearchGaps` (type: `boolean`):

Highlight areas that need more investigation

## `suggestFollowUpQuestions` (type: `boolean`):

Generate questions for deeper exploration based on research gaps

## Actor input object example

```json
{
  "topic": "artificial intelligence ethics",
  "outputFormats": [
    "json",
    "html",
    "pdf"
  ],
  "maxSourcesPerType": 10,
  "includeWikipedia": true,
  "includeAcademic": true,
  "includeNews": true,
  "includeGeneral": true,
  "searchProviders": [
    "duckduckgo"
  ],
  "requestTimeout": 30,
  "maxRetries": 3,
  "debug": false,
  "enableSmartCitations": true,
  "generateInsights": true,
  "identifyResearchGaps": true,
  "suggestFollowUpQuestions": true
}
```

# Actor output Schema

## `researchData` (type: `string`):

Structured research data with sources, AI analysis, and insights

## `generatedReports` (type: `string`):

Download JSON, HTML, and PDF research reports

## `fullOutput` (type: `string`):

Full research output including all sources, AI analysis, and processed content

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "topic": "artificial intelligence ethics",
    "outputFormats": [
        "json",
        "html",
        "pdf"
    ],
    "maxSourcesPerType": 10,
    "includeWikipedia": true,
    "includeAcademic": true,
    "includeNews": true,
    "includeGeneral": true,
    "searchProviders": [
        "duckduckgo"
    ],
    "requestTimeout": 30,
    "maxRetries": 3,
    "debug": false,
    "enableSmartCitations": true,
    "generateInsights": true,
    "identifyResearchGaps": true,
    "suggestFollowUpQuestions": true
};

// Run the Actor and wait for it to finish
const run = await client.actor("wheat_tourist/researchgpt-deep-research-agent").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "topic": "artificial intelligence ethics",
    "outputFormats": [
        "json",
        "html",
        "pdf",
    ],
    "maxSourcesPerType": 10,
    "includeWikipedia": True,
    "includeAcademic": True,
    "includeNews": True,
    "includeGeneral": True,
    "searchProviders": ["duckduckgo"],
    "requestTimeout": 30,
    "maxRetries": 3,
    "debug": False,
    "enableSmartCitations": True,
    "generateInsights": True,
    "identifyResearchGaps": True,
    "suggestFollowUpQuestions": True,
}

# Run the Actor and wait for it to finish
run = client.actor("wheat_tourist/researchgpt-deep-research-agent").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "topic": "artificial intelligence ethics",
  "outputFormats": [
    "json",
    "html",
    "pdf"
  ],
  "maxSourcesPerType": 10,
  "includeWikipedia": true,
  "includeAcademic": true,
  "includeNews": true,
  "includeGeneral": true,
  "searchProviders": [
    "duckduckgo"
  ],
  "requestTimeout": 30,
  "maxRetries": 3,
  "debug": false,
  "enableSmartCitations": true,
  "generateInsights": true,
  "identifyResearchGaps": true,
  "suggestFollowUpQuestions": true
}' |
apify call wheat_tourist/researchgpt-deep-research-agent --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,wheat_tourist/researchgpt-deep-research-agent"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/m1xxrwRUph2Rb8wLc/builds/aumrdfyEQ45j5Hbs0/openapi.json