🎯 TopicRadar - AI Research Assistant

Track any topic across the internet and get aggregated, ranked results from multiple sources in one place. Perfect for market research, competitive intelligence, trend monitoring, content creation, and staying updated on any subject.

✨ Features

• 🔥 Trending Topics: One-click access to curated trending topics in AI/ML, Startups, Development, and Research
• 🔍 Multi-Source Aggregation: Search across 7 reliable platforms: Hacker News, GitHub, arXiv, StackOverflow, Lobste.rs, Papers with Code, Semantic Scholar
• 🎯 Smart Ranking: Relevance-based, engagement-based, recency-based, or balanced ranking strategies
• 🧹 Intelligent Deduplication: Automatically removes duplicate content across sources
• 📊 Rich Metadata: Engagement metrics, publication dates, authors, and more
• 📝 Multiple Output Formats: JSON datasets or formatted Markdown reports
• ⚡ Fast & Parallel: Fetches from all sources simultaneously for quick results
• 🎛️ Highly Configurable: Filter by time range, engagement threshold, and more

🚀 Quick Start

Trending Topics (One-Click)

The easiest way to get started - just select a trending category:

{
  "searchMode": "trending-ai"
}

Available trending modes:

• trending-ai: AI agents, LLMs, GPT-4, Claude, machine learning, neural networks
• trending-startup: YC companies, startup funding, SaaS, bootstrapping, venture capital
• trending-dev: React, TypeScript, Rust, Kubernetes, DevOps, microservices
• trending-research: quantum computing, climate tech, biotech, fusion energy, CRISPR

Custom Topics Example

Or define your own topics to track:

{
  "searchMode": "custom",
  "topics": ["AI agents", "machine learning"],
  "sources": ["hackernews", "github", "arxiv", "stackoverflow", "lobsters", "paperswithcode", "semanticscholar"],
  "timeRange": "7d",
  "rankingStrategy": "balanced"
}

This will search for content about "AI agents" and "machine learning" from the last 7 days across all 7 sources, returning results ranked by a balanced algorithm.

Advanced Example

{
  "searchMode": "custom",
  "topics": ["climate tech", "carbon capture", "renewable energy"],
  "sources": ["hackernews", "github", "arxiv"],
  "timeRange": "30d",
  "maxResultsPerSource": 50,
  "rankingStrategy": "relevance",
  "minEngagementThreshold": 10,
  "includeSnippets": true,
  "outputFormat": "both"
}

📖 Use Cases

1. Quick Trend Check

Use trending topics for instant insights:

{
  "searchMode": "trending-startup"
}

2. Market Research

Track competitors, industry trends, and emerging technologies:

{
  "searchMode": "custom",
  "topics": ["your-competitor-name", "industry-trend"],
  "sources": ["hackernews", "github"],
  "timeRange": "7d"
}

3. Content Creation

Find trending topics and popular discussions for content ideas:

{
  "searchMode": "trending-dev",
  "rankingStrategy": "engagement"
}

4. Academic Research

Discover recent papers and discussions in your field:

{
  "searchMode": "custom",
  "topics": ["machine learning", "neural networks"],
  "sources": ["arxiv", "github"],
  "timeRange": "30d"
}

5. Weekly Tech Digest

Get a weekly overview of what's trending:

{
  "searchMode": "trending-ai",
  "timeRange": "7d",
  "minEngagementThreshold": 20
}

⚙️ Input Configuration

Parameter	Type	Required	Default	Description
searchMode	string	No	"trending-ai"	Search mode: trending categories or "custom"
topics	array	Only if custom	-	Keywords or phrases to track (1-10 items)
sources	array	No	All 5 sources	Content sources to search
timeRange	string	No	"7d"	Time period: "24h", "7d", or "30d"
maxResultsPerSource	integer	No	25	Maximum results per source (5-100)
rankingStrategy	string	No	"balanced"	Ranking method: "relevance", "engagement", "recent", or "balanced"
minEngagementThreshold	integer	No	5	Minimum engagement score to include results
includeSnippets	boolean	No	true	Include content preview snippets
outputFormat	string	No	"json"	Output format: "json", "markdown", or "both"

Available Sources

TopicRadar aggregates from 7 reliable platforms with official APIs:

• hackernews: Hacker News stories and discussions - Tech news and startup content
• github: GitHub repositories - Open source projects and code
• arxiv: arXiv research papers - Academic publications and scientific research
• stackoverflow: StackOverflow questions - Programming Q&A and technical discussions
• lobsters: Lobste.rs - Tech-focused community discussions
• paperswithcode: ML research papers with code implementations
• semanticscholar: Academic paper search engine with citation data

All sources use official APIs and are optimized for consistent performance and reliability.

Ranking Strategies

• relevance: Best keyword matches and topic alignment
• engagement: Most upvotes, comments, and reactions
• recent: Newest content first
• balanced: Combination of all factors (recommended)

📊 Output Format

JSON Dataset

Each result includes:

{
  "title": "Article or post title",
  "url": "https://example.com/article",
  "source": "hackernews",
  "author": "username",
  "publishedDate": "2026-01-15T10:30:00.000Z",
  "snippet": "Preview text from the content...",
  "relevanceScore": 85.5,
  "engagementMetrics": {
    "upvotes": 234,
    "comments": 56,
    "points": 234
  },
  "matchedTopics": ["AI agents", "LLM"],
  "metadata": {
    // Source-specific metadata
  }
}

Markdown Report

If outputFormat is set to "markdown" or "both", a formatted report will be saved to the Key-Value Store as report.md, including:

• Top 10 results with full details
• Results grouped by source
• Results grouped by topic
• Summary statistics

💡 Tips for Best Results

1. Be Specific with Topics: Use specific keywords rather than generic terms

   • ✅ Good: "React Server Components", "GPT-4 API"
   • ❌ Too broad: "programming", "AI"

2. Choose Relevant Sources: Different sources have different audiences

   • Dev content: hackernews, github, stackoverflow
   • Academic: arxiv, github
   • Tech discussions: hackernews, lobsters

3. Adjust Time Range: Balance freshness vs. quantity

   • Breaking news: 24h
   • Weekly trends: 7d
   • Deep research: 30d

4. Filter Noise: Use minEngagementThreshold to focus on quality

   • Popular content only: Set to 20-50
   • Include emerging content: Set to 5-10
   • Everything: Set to 0

5. Optimize Costs: Limit maxResultsPerSource if you want faster, cheaper runs

   • Quick scan: 10-15 per source
   • Thorough research: 50-100 per source

🔧 Advanced Usage

Schedule Regular Monitoring

Use Apify's scheduling feature to run TopicRadar daily or weekly:

1. Configure your input
2. Go to Actor Settings → Schedule
3. Set up cron schedule (e.g., 0 9 * * 1 for every Monday at 9 AM)

Integrate with Other Tools

Use Apify's webhooks to send results to:

• Slack/Discord for team notifications
• Google Sheets for tracking over time
• Your own API for custom processing
• Email for digest reports

Add GitHub Token for Better Rate Limits

If you're searching GitHub heavily, add your GitHub personal access token:

1. Go to Actor Settings → Environment Variables
2. Add GITHUB_TOKEN with your token
3. This increases rate limit from 60 to 5,000 requests/hour

🎯 Performance & Costs

• Runtime: Typically 30-90 seconds depending on sources and result count
• Memory: ~256MB (standard Actor)
• API Calls: All sources use free public APIs (except GitHub without token)
• Cost: Minimal - mostly platform compute time

Optimization Tips

• Use fewer sources for faster runs
• Lower maxResultsPerSource for cost savings
• Set appropriate minEngagementThreshold to filter noise
• Use timeRange: "24h" for quick daily scans

📈 Quality & Reliability

• ✅ Robust error handling - individual source failures won't crash the Actor
• ✅ Rate limiting protection - respects API limits automatically
• ✅ Deduplication - smart URL normalization and content matching
• ✅ Validated input/output schemas - ensures data quality
• ✅ Comprehensive logging - easy debugging and monitoring

🆘 Troubleshooting

"No results found"

• Try broader keywords
• Increase timeRange to 30d
• Lower minEngagementThreshold to 0
• Check if topics are spelled correctly

"GitHub rate limit exceeded"

• Add GITHUB_TOKEN environment variable
• Reduce number of topics
• Remove github from sources temporarily

"Actor timeout"

• Reduce maxResultsPerSource
• Use fewer sources
• Split topics across multiple runs

📝 Changelog

v1.0.0 (2026-01-20)

• 🔥 Trending Topics feature with 4 curated categories
• 🔍 Multi-source aggregation (5 reliable sources)
• 🎯 Multiple ranking strategies
• 📝 Markdown report generation
• 🧹 Smart deduplication
• ⚡ Optimized for reliability and performance

🤝 Support

Need help or have suggestions?

• Open an issue on GitHub
• Contact: mick.jae.johnson@gmail.com
• Check the Apify Actor documentation

📄 License

Apache 2.0

---

Built with ❤️ for the Apify $1M Challenge

Happy researching! 🚀 NEW - Broader Appeal Sources:

• techmeme: Tech news aggregator - what industry leaders are reading
• phnews: Product Hunt daily trending products and launches