Multi-Source Data API — GitHub, HN & Reddit 🚀

Apify Actor that fetches trending data from GitHub, Hacker News, and Reddit in one unified API call. Built for content curators, developer dashboard builders, news aggregators, and trend researchers.

⚡ Zero browser overhead — all data comes via free REST APIs, so runs are fast and cheap.

---

Features ✨

Feature	Description
3 Sources, 1 API	Fetch from GitHub Trending, Hacker News top stories, Reddit hot posts, or all at once
Smart Retry Logic	Automatic 3-attempt retry with exponential backoff on network errors and rate limits
Rate Limit Handling	Detects 429 responses and waits for Retry-After headers automatically
Parallel Fetching	When using all sources, fetches in parallel for maximum speed
Customizable Filters	Language filter for GitHub, subreddit selection for Reddit, time period for age
Input Validation	Validates all inputs before making API calls
Detailed Logging	Emoji-rich console output at every step for full transparency

---

Input Schema 📥

Parameter	Type	Default	Description
source	select	github	Data source: github, hackernews, reddit, or all
language	string	python	Programming language filter for GitHub trending
subreddit	string	programming	Subreddit to fetch hot posts from
maxResults	integer	15	Max items per source (1–100)
since	select	24h	Time period: 24h, 7d, or 30d

Example input

{
  "source": "all",
  "language": "typescript",
  "subreddit": "webdev",
  "maxResults": 10,
  "since": "7d"
}

---

Output Schema 📤

The actor pushes a single dataset entry with this structure:

{
  "source": "all",
  "totalResults": 30,
  "fetchedAt": "2026-07-17T12:00:00.000Z",
  "config": {
    "language": "typescript",
    "subreddit": "webdev",
    "maxResults": 10,
    "since": "7d"
  },
  "results": [
    {
      "_source": "github",
      "name": "vercel/ai",
      "stars": 12450,
      "forks": 1200,
      "description": "Build AI-powered applications...",
      "url": "https://github.com/vercel/ai",
      "language": "TypeScript",
      "topics": ["ai", "llm", "sdk"]
    },
    {
      "_source": "hackernews",
      "title": "Show HN: A new open-source database",
      "url": "https://example.com",
      "score": 342,
      "by": "username",
      "type": "story",
      "descendants": 87
    },
    {
      "_source": "reddit",
      "title": "What are you building?",
      "url": "https://reddit.com/r/webdev/...",
      "score": 156,
      "subreddit": "webdev",
      "comments": 43,
      "author": "user123"
    }
  ]
}

Result fields by source

Source	Key Fields
GitHub	name, stars, forks, description, url, language, topics, starsToday
Hacker News	title, url, score, by, type, descendants, time
Reddit	title, url, permalink, score, subreddit, comments, author, domain

All results include a _source field indicating the origin (github, hackernews, or reddit). When using all mode, results are sorted by score/stars descending.

---

Use Cases 🎯

1. Developer Dashboard Widget

Display trending GitHub repos, popular HN discussions, and active Reddit threads in a single developer dashboard panel.

// Input: {"source": "all", "language": "javascript", "maxResults": 5, "since": "24h"}

2. News & Content Aggregator

Build a daily digest of the most important developer news across all three platforms without managing separate integrations.

// Input: {"source": "all", "maxResults": 20, "since": "24h"}

3. Trend Monitoring Bot

Track what's gaining traction in a specific language ecosystem (e.g., Rust) by monitoring GitHub repos, HN discussions, and Reddit posts.

// Input: {"source": "all", "language": "rust", "subreddit": "rust", "maxResults": 10, "since": "7d"}

4. Social Proof & Link Curation

Find high-engagement content for newsletters, social media posts, or blog references by scoring across platforms.

// Input: {"source": "hackernews", "maxResults": 30, "since": "24h"}

5. Competitive Research

Track what open-source projects are trending in your competitor's tech stack.

// Input: {"source": "github", "language": "python", "maxResults": 25, "since": "7d"}

6. Multilingual Community Monitoring

Monitor niche subreddits and language-specific GitHub repos for emerging technologies.

// Input: {"source": "all", "language": "kotlin", "subreddit": "androiddev", "maxResults": 10}

---

FAQ ❓

Q: Why is Reddit returning no data?

A: As of mid-2023, Reddit restricts API access from many cloud IP ranges (including Apify's). The Reddit source tries multiple endpoints but will gracefully fail with a clear message if blocked. To use Reddit data reliably, consider setting up a Reddit OAuth app or running the actor from a residential IP.

Q: Is this actor free to use?

A: Yes! All three data sources (GitHub, HN, Reddit) use free public REST APIs. You only pay for Apify platform compute time, which is minimal since there's no browser overhead.

Q: How many results can I fetch?

A: Up to 100 per source. Using all mode with maxResults: 100 fetches up to 300 total items (100 from each source).

Q: What happens if one source fails?

A: In all mode, each source is fetched independently. If one fails (e.g., GitHub rate limited), the other two still return data. Failures are logged clearly.

Q: Does the actor handle API rate limits?

A: Yes! It automatically detects 429 (Too Many Requests) responses and respects Retry-After headers with exponential backoff. GitHub rate limit remaining is logged after each request.

Q: What time period options are available?

A: 24h (last 24 hours), 7d (last 7 days), or 30d (last 30 days). This primarily affects GitHub's search query; HN and Reddit always return their current top/hot lists.

Q: Can I run this on a schedule?

A: Absolutely. Set up an Apify schedule (e.g., every 6 hours) to build a continuous data pipeline for dashboards or monitoring.

Q: What's in the _source field?

A: Every result includes _source set to "github", "hackernews", or "reddit" so you can easily filter or tag items by origin in downstream processing.

---

Pricing 💰

This actor uses free REST APIs — no paid API keys required. Apify platform charges are based on compute time only, which is typically under 3 seconds per run (~$0.001 USD per run on Apify). Scheduled runs at any interval are cost-effective.

---

Technical Notes 🔧

• Runtime: Node.js 20 (Playwright image — retained for compatibility, but Playwright is not used)
• Retry Strategy: 3 attempts with exponential backoff (1s → 2s → 4s, capped at 15s)
• Timeout: 30s per individual request
• Edge Cases Handled:
  • Empty responses from any source
  • Network timeouts and connection resets
  • Rate limiting (429) with Retry-After support
  • Missing fields in API responses (graceful fallback to defaults)
  • Invalid subreddit names (returns empty array)
  • Malformed JSON responses (caught and logged)
  • Partial failures in all mode (other sources still return data)

---

Built with ❤️ for the Dollar Hunter project — set & forget income through automation.