Hacker News Scraper API + AI Thread Digest

Hacker News Scraper at a glance: data from the official Hacker News APIs (Firebase and Algolia) · no login, no browser, no proxies · $1 per 1,000 stories · optional AI with your own key · JSON, CSV, Excel export, API and MCP ready

What does the Hacker News Scraper do?

Scrape Hacker News stories from the front page, Show HN, Ask HN, best and newest, or run a keyword search with date filters. Each story comes with points, comment counts, URLs and the top comments' text. Turn on the optional AI digest to get a summary, sentiment, key points, mentioned products and a controversy score for every discussion.

The Actor uses the official Hacker News APIs (Firebase and Algolia), so there is no HTML scraping, no blocking and no browser.

Why this Hacker News scraper?

• Every HN feed in one Actor: front_page, best, new, show_hn, ask_hn and full-text search (by relevance or date)
• Filters: date range, minimum points, minimum comments
• Comment text, without usernames: top comments are fetched breadth-first, so you get the best of the discussion and no user profiles
• AI thread digest: turn a 400-comment thread into a 5-line brief, which works well for newsletters and market research
• Fast and cheap: pure API calls

Use cases

• Brand and competitor monitoring: track every HN mention of your product, with sentiment
• Tech trend research: see what developers think about a framework, AI model or startup
• Newsletters and digests: a daily or weekly "best of HN" with AI summaries
• Launch tracking: monitor Show HN launches in your category
• AI and RAG datasets: high-signal technical discussion for LLM pipelines

How to use the Hacker News Scraper

1. Click Try for free (or Start) on this page and sign in to Apify.
2. Pick a mode: front_page, best, new, show_hn, ask_hn or keyword search.
3. Optionally set a date range, minimum points or comments, and whether to include top comment text.
4. Optionally enable the AI thread digest and paste your OpenAI or OpenRouter key.
5. Click Start, then download results as JSON, CSV or Excel, or fetch them via the API.

Input

Field	Description
mode	front_page, search, show_hn, ask_hn, best or new
query	Search keywords (for search mode)
sortBy	relevance or date
dateFrom / dateTo	Date range filter (YYYY-MM-DD)
minPoints / minComments	Quality filters
maxStories	Maximum number of stories (default 50)
includeComments, maxCommentsPerStory	Fetch top comment text (default 30 per story)
enableAi, llmProvider, llmApiKey, llmModel	Optional AI digest with your own key

{ "mode": "search", "query": "web scraping", "minPoints": 50, "maxStories": 20, "includeComments": true }

Output example

One row per story, straight from the official Hacker News APIs (Firebase and Algolia):

{
  "id": "22180559",
  "title": "Congrats! Web scraping is legal! (US precedent)",
  "url": "https://parsers.me/us-court-fully-legalized-website-scraping-and-technically-prohibited-it/",
  "domain": "parsers.me",
  "points": 1057,
  "numComments": 388,
  "createdAt": "2020-01-29T14:06:37Z",
  "type": "story",
  "hnUrl": "https://news.ycombinator.com/item?id=22180559",
  "text": null,
  "topComments": [
    {
      "text": "Finally some common sense. I’m also somewhat impressed that it “only” took 3 years from the initial lawsuit to reaching this precedent. I expected this to go...",
      "depth": 0,
      "createdAt": "2020-01-29T14:17:42.000Z"
    },
    {
      "text": "IANAL - but also ran afoul of LinkedIns data scraping blocks (and got a cease and desist letter) but what they said was it was a violation of the TOS. This d...",
      "depth": 0,
      "createdAt": "2020-01-29T14:18:35.000Z"
    }
  ],
  "scrapedAt": "2026-10-31T15:03:23.399Z"
}

With AI turned on, each story also gets an ai object: summary, sentiment, keyPoints[], mentionedProducts[], category and controversyScore.

How much does it cost?

This Actor uses pay per event pricing: you pay only for results saved to your dataset, with no monthly fee. Apify platform compute is included.

Event	Price
Story scraped	$0.001 per story ($1 per 1,000)
AI thread digest	$0.003 per digested story (plus your own LLM tokens)

Examples

• 1,000 stories: $1
• 10,000 stories: $10
• 1,000 stories with AI: $4 plus your LLM tokens (typically cents with gpt-4o-mini)

Apify's Free plan includes $5 of monthly usage, enough for about 5,000 stories without AI. Set Maximum cost per run in the run options to cap spend; the Actor stops cleanly at the limit.

Use it via API, Python, n8n, Make or AI agents

Run the Actor and get results in one call (replace <APIFY_TOKEN> with your token):

curl -X POST "https://api.apify.com/v2/acts/data-drip~hacker-news-ai-digest/run-sync-get-dataset-items?token=<APIFY_TOKEN>" \
  -H 'Content-Type: application/json' \
  -d '{"mode": "search", "query": "web scraping", "minPoints": 50, "maxStories": 20, "includeComments": true}'

from apify_client import ApifyClient

client = ApifyClient('<APIFY_TOKEN>')
run = client.actor('data-drip/hacker-news-ai-digest').call(run_input={"mode": "search", "query": "web scraping", "minPoints": 50, "maxStories": 20, "includeComments": True})
for item in client.dataset(run['defaultDatasetId']).iterate_items():
    print(item)

• Schedules: run it hourly, daily or weekly from Apify Schedules.
• Integrations: send results to Google Sheets, Slack, Zapier, Make, n8n or a webhook.
• AI agents (MCP): add data-drip/hacker-news-ai-digest as a tool in the Apify MCP server (https://mcp.apify.com) for Claude, Cursor and other MCP clients.

Limitations

• Usernames and user profiles are deliberately not collected.
• Comment text is limited to the top comments per story (configurable), not full threads.
• Points and comment counts are a snapshot at run time.

FAQ

Is it allowed? Yes. Hacker News provides official public APIs for programmatic access. The Actor stores no usernames or profiles, only story metadata and comment text.

How far back can I search? Algolia search covers the whole history of HN, back to 2007.

Do I need an AI key? No. Without a key the Actor runs as a fast, plain scraper and you pay only for results. To turn on AI, paste your own OpenAI or OpenRouter key into the LLM API key field. It is stored as an encrypted secret input and only used to call the provider you picked. LLM tokens are billed by your provider, not by us.

Which models work? The default is gpt-4o-mini, or openai/gpt-4o-mini on OpenRouter. Any chat model that supports JSON output works. Set it in LLM model.

Can I schedule it? Yes. Use Apify Schedules, for example daily front_page with minPoints: 100, and send the results to Slack, email or Google Sheets.

How do I get help or request a feature? Open an issue on the Actor's Issues tab.

---

Related searches: hacker news scraper, hacker news api, ycombinator news scraper, show hn, ask hn, hn comments, brand monitoring, tech news scraper, sentiment analysis, ai summaries.