🔬 ResearchGPT - Deep Research Agent

Transform any topic into a comprehensive research report in minutes, not hours.

---

🎯 What is ResearchGPT?

ResearchGPT is your AI-powered research assistant that does in 3 minutes what would take you 3+ hours manually.

Simply enter any topic, and ResearchGPT will:

✅ Search across multiple engines (DuckDuckGo, Brave, Mojeek)
✅ Scrape Wikipedia, arXiv, Semantic Scholar, OpenAlex, CrossRef
✅ Extract the latest news articles and web content
✅ Process everything with intelligent NLP analysis
✅ Generate beautiful reports in JSON, HTML & PDF formats

No API keys required. No complex setup. Just results.

---

🚀 Perfect For

Use Case	How ResearchGPT Helps
📚 Students & Academics	Literature reviews, thesis research, citation gathering
✍️ Content Creators	Blog research, fact-checking, source compilation
💼 Business Analysts	Market research, competitive analysis, trend reports
🔬 Researchers	Cross-referencing sources, academic paper aggregation
📰 Journalists	Background research, source verification, story development
🤖 AI/ML Projects	Training data collection, knowledge base building

---

⚡ Quick Start

1. Run on Apify (Easiest)

1. Go to the ResearchGPT Actor page
2. Enter your research topic
3. Click Start
4. Download your reports! 📄

2. Via API

curl -X POST "https://api.apify.com/v2/acts/YOUR_USERNAME~researchgpt-deep-research-agent/runs?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"topic": "quantum computing breakthroughs 2025"}'

3. Via Apify SDK (Python)

from apify_client import ApifyClient

client = ApifyClient("YOUR_API_TOKEN")

run = client.actor("YOUR_USERNAME/researchgpt-deep-research-agent").call(
    run_input={"topic": "artificial intelligence in healthcare"}
)

# Get results
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

---

📊 What You Get

Three Professional Output Formats

Format	Best For	Contents
📄 JSON	Developers, APIs, databases	Full structured data with metadata
🌐 HTML	Web publishing, sharing	Beautifully styled report with CSS
📑 PDF	Printing, presentations	Clean, professional document

Rich Research Data

{
  "topic": "artificial intelligence ethics",
  "sources": {
    "wikipedia": 5,
    "academic": 10,
    "news": 5,
    "general": 10
  },
  "processed_content": {
    "summary": "Comprehensive executive summary...",
    "key_findings": ["Finding 1", "Finding 2", "..."],
    "themes": ["Theme 1", "Theme 2", "..."],
    "entities": ["Entity 1", "Entity 2", "..."]
  }
}

---

🔧 Configuration Options

{
  "topic": "your research topic here",
  "outputFormats": ["json", "html", "pdf"],
  "maxSourcesPerType": 10,
  "includeWikipedia": true,
  "includeAcademic": true,
  "includeNews": true,
  "includeGeneral": true,
  "searchProviders": ["duckduckgo"],
  "requestTimeout": 30,
  "maxRetries": 3,
  "debug": false
}

Parameter Reference

Parameter	Type	Default	Description
topic	string	required	🎯 Your research topic or question
outputFormats	array	["json", "html", "pdf"]	📄 Output formats to generate
maxSourcesPerType	integer	10	📊 Sources per category (1-20)
includeWikipedia	boolean	true	📖 Include Wikipedia articles
includeAcademic	boolean	true	🎓 Include academic papers
includeNews	boolean	true	📰 Include news articles
includeGeneral	boolean	true	🌐 Include general web content
searchProviders	array	["duckduckgo"]	🔍 Search engines to use
requestTimeout	integer	30	⏱️ Request timeout (seconds)
maxRetries	integer	3	🔄 Retry attempts on failure
proxyConfiguration	object	null	🛡️ Apify proxy settings
debug	boolean	false	🐛 Enable verbose logging

---

🌐 Data Sources

ResearchGPT taps into 6+ authoritative sources:

Source	Type	What You Get
🌍 Wikipedia	Knowledge Base	Foundational articles via MediaWiki API
📚 arXiv	Academic	Pre-print papers in physics, CS, math, and more
🔬 Semantic Scholar	Academic	200M+ papers with citation analysis
📖 OpenAlex	Academic	Open catalog of scholarly works
📑 CrossRef	Academic	DOI metadata and citations
📰 News Sources	Current Events	Latest articles via smart extraction
🌐 General Web	Insights	Curated web content with readability algorithms

---

🛡️ Production-Grade Features

Built for reliability and scale:

• ⚡ Smart Caching - 5-minute TTL prevents redundant requests
• 🔄 Retry Logic - Exponential backoff with jitter
• 🚦 Rate Limiting - Respects API limits automatically
• 🔍 Deduplication - MD5-based content fingerprinting
• 🌐 Connection Pooling - Efficient HTTP management
• 🛡️ Error Handling - Graceful fallbacks, never crashes

---

💻 Local Development

# Clone the repository
git clone https://github.com/your-repo/researchgpt-deep-research-agent
cd researchgpt-deep-research-agent

# Create virtual environment
python -m venv .venv
.venv\Scripts\activate      # Windows
source .venv/bin/activate   # macOS/Linux

# Install dependencies
pip install -r requirements.txt

# Run locally
python run_local.py

---

📁 Project Structure

researchgpt-deep-research-agent/
├── 📂 .actor/
│   └── actor.json           # Apify configuration
├── 📂 src/
│   ├── __init__.py
│   └── main.py              # 🚀 Main Apify entry point
├── 📂 scrapers/
│   ├── base_scraper.py      # Base class with retry/caching
│   ├── academic_scraper.py  # arXiv, Semantic Scholar, etc.
│   ├── wikipedia_scraper.py # MediaWiki API
│   ├── news_scraper.py      # News extraction
│   ├── general_scraper.py   # Web scraping
│   └── search_engine.py     # Multi-provider search
├── 📂 processors/
│   └── content_processor.py # NLP processing
├── 📂 output/
│   └── output_generator.py  # Report generation
├── 📄 Dockerfile            # Container definition
├── 📄 requirements.txt      # Dependencies
└── 📄 README.md             # You are here!

---

🚀 Deploy to Apify

Option 1: Apify CLI (Recommended)

npm install -g apify-cli
apify login
apify push

Option 2: GitHub Integration

1. Push to GitHub
2. Apify Console → Create Actor → Link to GitHub
3. Auto-builds on every push! 🔄

---

📈 Performance Tips

Tip	Impact
Lower maxSourcesPerType	⚡ Faster results
Disable unused sources	🚀 Skip what you don't need
Use single search provider	📉 Reduce API calls
Enable debug mode	🔍 Troubleshoot issues

---

🤔 FAQ

---

🆚 Why ResearchGPT?

Feature	ResearchGPT	Manual Research	Other Tools
⏱️ Time	3 minutes	3+ hours	30+ minutes
📚 Sources	6+ databases	Limited	Usually 1-2
📄 Output	JSON + HTML + PDF	Manual formatting	Single format
💰 Cost	Pay per run	Your time = $$$$	Subscription
🔧 Setup	Zero	N/A	API keys needed

---

📝 Example Topics

Get inspired! Here are some topics that work great:

• "artificial intelligence ethics and regulation 2025"
• "quantum computing practical applications"
• "climate change solutions renewable energy"
• "cryptocurrency DeFi market analysis"
• "remote work productivity research"
• "mental health digital therapeutics"
• "gene editing CRISPR medical applications"
• "electric vehicles battery technology"

---

🤝 Support & Community

• 🐛 Issues: Report bugs
• 💡 Feature Requests: Suggest ideas
• 📖 Docs: Apify Documentation
• 💬 Discord: Join Apify Community

---

📄 License

MIT License - Use it freely, commercially or personally.

---