Hacker News Scraper — Stories, Comments & Profiles

What does Hacker News Scraper do?

Hacker News Scraper enables you to extract data from Hacker News using the public Firebase API. No login or authentication required.

Hacker News Scraper can scrape:

• Stories from any of the six HN feeds (top, new, best, ask, show, job)
• Full comment trees for each story
• Submitter profiles (karma, account age, bio)
• Story metadata (score, comments count, timestamps, URLs)
• Comment metadata (author, timestamp, parent/child relationships)

Why scrape Hacker News?

Hacker News is one of the most influential tech communities online. It's an excellent source of data for AI training, market research, and competitive intelligence.

Here are just some of the ways you could use Hacker News data:

• Build training datasets for tech-aware language models
• Track which products, libraries, and frameworks are gaining traction
• Research founders and their posting patterns before investing
• Monitor trends in specific HN categories (Ask HN, Show HN, jobs)
• Build RSS-like feeds of curated HN content for newsletters
• Analyze comment sentiment and engagement on tech discussions

If you would like more inspiration on how scraping Hacker News could help your business or organization, check out our industry pages.

How to scrape Hacker News

It's easy to scrape Hacker News with Hacker News Scraper. Just follow these few steps and you'll get your data in minutes.

1. Click on Try for free.
2. Select a feed mode (Top Stories is the default).
3. Configure the max number of stories and whether to include comments and submitter profiles.
4. Click on Run.
5. When Hacker News Scraper has finished, preview or download your data from the Dataset tab.

How much will it cost to scrape Hacker News?

Apify gives you $5 free usage credits every month on the Apify Free plan. A typical run of 100 stories costs $0.50 — well within the free tier.

But if you need to scrape Hacker News regularly or at scale, we recommend our $49/month Starter plan for consistent access to fresh data.

For large-scale scraping operations, the Scale plan offers even more value with higher monthly allowances.

Results

The actor returns data in JSON format, one record per story:

{
  "type": "story",
  "id": 42345678,
  "title": "Show HN: I built a thing",
  "url": "https://example.com",
  "score": 234,
  "by": "username",
  "time": 1718371200,
  "timeIso": "2024-06-14T10:00:00Z",
  "descendants": 87,
  "commentCount": 87,
  "type_hn": "story",
  "submitter": {
    "id": "username",
    "created": 1234567890,
    "createdIso": "2009-02-13T23:31:30Z",
    "karma": 5678,
    "about": null
  },
  "comments": [
    {
      "id": 42345679,
      "by": "commenter1",
      "text": "Great work!",
      "time": 1718371300,
      "timeIso": "2024-06-14T10:01:40Z",
      "deleted": false,
      "dead": false,
      "parent": 42345678
    }
  ]
}

Tips for scraping Hacker News

• The Firebase API is free and has no rate limits — runs are fast (50+ concurrent fetches)
• For large comment threads, lower maxCommentsPerStory to keep runs bounded
• Disable fetchProfiles for cheaper runs that only need story metadata
• The job_stories and ask_stories modes are great for niche research
• Combine with the Story URL to link directly back to HN

Is it legal to scrape Hacker News?

Note that personal data is protected by GDPR in the European Union and by other regulations around the world. Hacker News data is public, but you should not scrape personal data unless you have a legitimate reason to do so. If you're unsure whether your reason is legitimate, consult your lawyers. We also recommend that you read our blog post: is web scraping legal?.