Hacker News Scraper avatar

Hacker News Scraper

Pricing

from $0.10 / 1,000 results

Go to Apify Store
Hacker News Scraper

Hacker News Scraper

Scrape Hacker News stories, comments, and user data. Supports top/new/best/ask/show/job story feeds and full-text keyword search via the Algolia API. Extract titles, URLs, scores, authors, comment counts, and timestamps.

Pricing

from $0.10 / 1,000 results

Rating

0.0

(0)

Developer

Monkey Coder

Monkey Coder

Maintained by Community

Actor stats

1

Bookmarked

252

Total users

63

Monthly active users

16 days ago

Last modified

Security

Limited permissions

Share

Collect Hacker News stories, bounded comment trees, user submissions, and recent discussion trends without authentication.

Modes

  • top, new, best, ask, show, job: official Firebase feeds.
  • search: Algolia full-text story search.
  • user: stories submitted by one HN user.
  • story_comments: fetch a story and its comment tree from IDs or HN URLs.
  • trends: aggregate recent Algolia stories into keyword rows with story count, points, comments, domains, and an explainable trend_score.

Comment enrichment

Set include_comments for feed/search/user stories. comment_limit bounds requests per story, comment_depth controls reply levels, and comment_sort can preserve HN rank or sort by newest/score. Every comment includes id, author, text, score, parent_id, depth, kids_count, created_at, hn_url, deleted, and dead. top_comments remains as a compatibility alias for the new comments field.

Trend example

{
"mode": "trends",
"trend_queries": ["AI", "security", "developer tools"],
"hours": 72,
"max_results": 50
}

Trend rows are ranked using a transparent blend of unique story count, total points, and total comments. This makes the output useful for topic monitoring, editorial research, and product discovery while avoiding an opaque model score.

Performance and reliability

Firebase calls use a shared requests.Session, retry 429 and 5xx responses with bounded exponential backoff and Retry-After, and cap comment traversal to prevent unbounded recursion. Invalid inputs produce one structured error record instead of placeholder duplicates.