🚀 Y Combinator News Scraper

Want the newest submissions from Hacker News plus the main article text from each linked source? This actor does both in one run.

It always starts from HN’s newest feed. There it reads titles, scores, authors, and each story’s outbound link. Then it opens those destination pages and extracts readable body text when available. Metadata comes from the HN listing; content comes from the linked source.

💡 Perfect for…

• Researchers & analysts: Track what’s being submitted and pull readable text from the original page.
• Newsletters & dashboards: Combine HN metadata (points, author, hn_discuss_url) with article excerpts.
• 📚 RAG systems: Index title, news_link, and content so answers can cite both HN context and the source article.

✨ Why you'll love this scraper

• 🧹 Clean saves: Rows are stored only when usable body text was found — empty or blocked pages are skipped.
• 👤 Structured HN fields: Ids, title, outbound link, site label, score, author, timestamps, discussion URL, plus content.

📦 What's inside the data?

For every story that yields extractable text:

• From HN: id, title, news_link, site_domain, points, author, posted_at_iso, posted_at_human, hn_discuss_url
• From the linked page: content (plain article text when extraction succeeds)

🚀 Quick start

1. Set max_news (how many newest stories to try, up to 200).
2. Start the actor — no listing URL to paste; the feed is fixed.
3. Export the dataset as CSV, Excel, or JSON when finished.

---

Example input

{
  "max_news": 100
}

Example output

{
  "id": "47824343",
  "title": "HTTP11Probe – Probe web frameworks for compliance",
  "news_link": "https://www.http-probe.com/",
  "site_domain": "http-probe.com",
  "points": 1,
  "author": "MDA2AV",
  "posted_at_iso": "2026-04-19T13:55:47",
  "posted_at_human": "1 minute ago",
  "hn_discuss_url": "https://news.ycombinator.com/item?id=47824343",
  "content": "An open testing platform that probes HTTP/1.1 servers…"
}

Parameter	Type	Required	Description
max_news	integer	No	How many newest stories to collect (and open for content). Default 100, min 20, max 200.