AI Agent News Aggregator - GitHub, HN & X Multi-Source Feed

One API call to fetch real-time AI Agent news from GitHub, Hacker News, and X (Twitter). Quality-ranked by exponential decay scoring, cross-source deduplicated, structured JSON output — instantly ready for AI Agents, content creators, and researchers.

---

Why This Actor?

Feature	This Actor	Single-Source Scrapers
Data Sources	3 (GitHub + HN + X)	1
Quality Ranking	✅ Exponential decay (12h half-life)	❌ Chronological only
Cross-Source Dedup	✅ URL normalization + MD5 fingerprint	❌
Zero Setup	✅ GitHub & HN work instantly	Varies
AI-Ready Output	✅ Structured JSON with engagement metrics	Raw data
x402 Micropayment	✅ Agent-to-Agent billing ready	❌

---

Use Cases

• AI Agent Developers — Feed real-time industry news to your agent's knowledge base
• Content Creators — Track trending AI topics across platforms for content ideas
• Researchers — Monitor AI Agent ecosystem developments and sentiment
• Investors — Track GitHub stars, HN discussions, and X conversations

---

⚡ Quick Start (No Setup Required)

1. Click "Run" with default settings → instant GitHub + HN data (no API keys needed)
2. (Optional) To enable X (Twitter), paste your X session cookies into twitterCookies
3. Coming soon: Reddit News & Business Subscription!

---

📋 Input Presets

Preset A: Zero-Setup Mode (Fastest)

{
  "keywords": ["AI Agent", "LLM"],
  "sources": ["hackernews", "github"],
  "timeRange": "24h",
  "maxItems": 30
}

Preset B: Full Cross-Platform Mode (Includes X)

Requires X burner account cookies (see How to Get Cookies):

{
  "keywords": ["AI Agent", "Claude Code"],
  "sources": ["hackernews", "github", "twitter"],
  "timeRange": "7d",
  "maxItems": 50,
  "twitterCookies": "[{\"name\":\"auth_token\",\"value\":\"...\"},{\"name\":\"ct0\",\"value\":\"...\"}]"
}

---

🛠️ Input Parameters

Parameter	Type	Default	Description
keywords	array	["AI Agent"]	Search keywords. Supports English/Chinese mixed terms.
sources	array	["hackernews","github"]	Platforms: hackernews, github, twitter.
timeRange	string	"24h"	Time horizon: "1h", "6h", "24h", "7d".
maxItems	integer	30	Max deduplicated items in output.
twitterCookies	string	""	Optional. X session cookies (JSON array or Netscape .txt).

---

📊 Output Schema

Unified output with quality scoring across all sources:

{
  "id": "github_1223170290",
  "source": "github",
  "title": "open-design",
  "url": "https://github.com/nexu-io/open-design",
  "author": { "handle": "nexu-io", "followers": 0 },
  "content": "The open-source Claude Design alternative...",
  "engagement": { "likes": 80235, "retweets": 9251, "replies": 0, "views": 0, "comments": 0 },
  "metrics": { "score": 108354.28, "engagementRate": 0.0, "viralityScore": 0 },
  "publishedAt": "2026-07-21T12:00:19Z",
  "fetchedAt": "2026-07-21T12:02:46Z"
}

Score Formula: S = Engagement × e^(-λ × hours_old) × SourceWeight

• λ = ln(2) / 12 (12-hour half-life)
• Source weights: HN 1.5×, Twitter 1.2×, GitHub 1.1×

---

🍪 How to Get X/Twitter Cookies

⚠️ WARNING: Always use a burner/alt X account. Never use your personal main account. Automated scraping may trigger X's anti-abuse system.

Since auth_token is HttpOnly, you must use a browser extension:

1. Log into x.com with your burner account
2. Install a cookie export extension:
   • Cookie-Editor (Chrome/Firefox/Edge) → Export as JSON
   • Get cookies.txt (Chrome/Firefox) → Export as .txt
3. Paste the exported content into twitterCookies

The Actor auto-parses both JSON arrays and Netscape .txt formats.

Cookies expire in 2-4 weeks. Re-export when you see 401 or 403 errors.

---

🛡️ Anti-Detection (X/Twitter)

Protection	Implementation
Sticky Session Proxy	Fixed residential IP per account
Error Recovery	Auto-detect 401/403/rate-limit, graceful degradation

---

💰 Pricing & Billing

All resource consumption (Compute Units + Apify Residential Proxy) is deducted from your Apify subscription credits. No additional fees until credits are consumed.

---

📈 Performance

Source	Items per Query	Latency
GitHub	50-100 repos	~5s
Hacker News	30-60 stories	~3s
X/Twitter	10-30 tweets	~15s
Full 3-Source	100-200 items	< 30s

Cross-source dedup typically removes 20-40% overlap.

---

🔗 Related

• Apify Actor Page