# Hacker News Scraper API + AI Thread Digest (`data-drip/hacker-news-ai-digest`) Actor

Scrape Hacker News stories (front page, Show HN, Ask HN, best, newest or keyword search by date) with points, comment counts and top comment text via the official HN APIs. Optional AI digest per thread: summary, sentiment, key points, products.

- **URL**: https://apify.com/data-drip/hacker-news-ai-digest.md
- **Developed by:** [Ibraheem Ganayim](https://apify.com/data-drip) (community)
- **Categories:** News, AI, Developer tools
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.00 / 1,000 story scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Hacker News Scraper API + AI Thread Digest

**Hacker News Scraper at a glance:** data from the official Hacker News APIs (Firebase and Algolia) · no login, no browser, no proxies · **$1 per 1,000 stories** · optional AI with your own key · JSON, CSV, Excel export, API and MCP ready

### What does the Hacker News Scraper do?

Scrape **Hacker News** stories from the front page, **Show HN**, **Ask HN**, best and newest, or run a **keyword search with date filters**. Each story comes with points, comment counts, URLs and the **top comments' text**. Turn on the optional **AI digest** to get a summary, sentiment, key points, mentioned products and a controversy score for every discussion.

The Actor uses the **official Hacker News APIs** (Firebase and Algolia), so there is no HTML scraping, no blocking and no browser.

### Why this Hacker News scraper?

- **Every HN feed in one Actor**: `front_page`, `best`, `new`, `show_hn`, `ask_hn` and full-text `search` (by relevance or date)
- **Filters**: date range, minimum points, minimum comments
- **Comment text, without usernames**: top comments are fetched breadth-first, so you get the best of the discussion and no user profiles
- **AI thread digest**: turn a 400-comment thread into a 5-line brief, which works well for newsletters and market research
- **Fast and cheap**: pure API calls

### Use cases

- **Brand and competitor monitoring**: track every HN mention of your product, with sentiment
- **Tech trend research**: see what developers think about a framework, AI model or startup
- **Newsletters and digests**: a daily or weekly "best of HN" with AI summaries
- **Launch tracking**: monitor Show HN launches in your category
- **AI and RAG datasets**: high-signal technical discussion for LLM pipelines

### How to use the Hacker News Scraper

1. Click **Try for free** (or **Start**) on this page and sign in to Apify.
2. Pick a mode: `front_page`, `best`, `new`, `show_hn`, `ask_hn` or keyword `search`.
3. Optionally set a date range, minimum points or comments, and whether to include top comment text.
4. Optionally enable the AI thread digest and paste your OpenAI or OpenRouter key.
5. Click **Start**, then download results as JSON, CSV or Excel, or fetch them via the API.

### Input

| Field | Description |
|---|---|
| `mode` | `front_page`, `search`, `show_hn`, `ask_hn`, `best` or `new` |
| `query` | Search keywords (for `search` mode) |
| `sortBy` | `relevance` or `date` |
| `dateFrom` / `dateTo` | Date range filter (YYYY-MM-DD) |
| `minPoints` / `minComments` | Quality filters |
| `maxStories` | Maximum number of stories (default 50) |
| `includeComments`, `maxCommentsPerStory` | Fetch top comment text (default 30 per story) |
| `enableAi`, `llmProvider`, `llmApiKey`, `llmModel` | Optional AI digest with your own key |

```json
{ "mode": "search", "query": "web scraping", "minPoints": 50, "maxStories": 20, "includeComments": true }
```

### Output example

One row per story, straight from the official Hacker News APIs (Firebase and Algolia):

```json
{
  "id": "22180559",
  "title": "Congrats! Web scraping is legal! (US precedent)",
  "url": "https://parsers.me/us-court-fully-legalized-website-scraping-and-technically-prohibited-it/",
  "domain": "parsers.me",
  "points": 1057,
  "numComments": 388,
  "createdAt": "2020-01-29T14:06:37Z",
  "type": "story",
  "hnUrl": "https://news.ycombinator.com/item?id=22180559",
  "text": null,
  "topComments": [
    {
      "text": "Finally some common sense. I’m also somewhat impressed that it “only” took 3 years from the initial lawsuit to reaching this precedent. I expected this to go...",
      "depth": 0,
      "createdAt": "2020-01-29T14:17:42.000Z"
    },
    {
      "text": "IANAL - but also ran afoul of LinkedIns data scraping blocks (and got a cease and desist letter) but what they said was it was a violation of the TOS. This d...",
      "depth": 0,
      "createdAt": "2020-01-29T14:18:35.000Z"
    }
  ],
  "scrapedAt": "2026-10-31T15:03:23.399Z"
}
```

With AI turned on, each story also gets an `ai` object: `summary`, `sentiment`, `keyPoints[]`, `mentionedProducts[]`, `category` and `controversyScore`.

### How much does it cost?

This Actor uses **pay per event** pricing: you pay only for results saved to your dataset, with no monthly fee. Apify platform compute is included.

| Event | Price |
|---|---|
| Story scraped | **$0.001** per story ($1 per 1,000) |
| AI thread digest | **$0.003** per digested story (plus your own LLM tokens) |

**Examples**

- 1,000 stories: **$1**
- 10,000 stories: **$10**
- 1,000 stories with AI: **$4** plus your LLM tokens (typically cents with `gpt-4o-mini`)

Apify's Free plan includes $5 of monthly usage, enough for about **5,000 stories** without AI. Set **Maximum cost per run** in the run options to cap spend; the Actor stops cleanly at the limit.

### Use it via API, Python, n8n, Make or AI agents

Run the Actor and get results in one call (replace `<APIFY_TOKEN>` with your token):

```bash
curl -X POST "https://api.apify.com/v2/acts/data-drip~hacker-news-ai-digest/run-sync-get-dataset-items?token=<APIFY_TOKEN>" \
  -H 'Content-Type: application/json' \
  -d '{"mode": "search", "query": "web scraping", "minPoints": 50, "maxStories": 20, "includeComments": true}'
```

```python
from apify_client import ApifyClient

client = ApifyClient('<APIFY_TOKEN>')
run = client.actor('data-drip/hacker-news-ai-digest').call(run_input={"mode": "search", "query": "web scraping", "minPoints": 50, "maxStories": 20, "includeComments": True})
for item in client.dataset(run['defaultDatasetId']).iterate_items():
    print(item)
```

- **Schedules**: run it hourly, daily or weekly from Apify Schedules.
- **Integrations**: send results to Google Sheets, Slack, Zapier, Make, n8n or a webhook.
- **AI agents (MCP)**: add `data-drip/hacker-news-ai-digest` as a tool in the Apify MCP server (`https://mcp.apify.com`) for Claude, Cursor and other MCP clients.

### Limitations

- Usernames and user profiles are deliberately not collected.
- Comment text is limited to the top comments per story (configurable), not full threads.
- Points and comment counts are a snapshot at run time.

### FAQ

**Is it allowed?** Yes. Hacker News provides official public APIs for programmatic access. The Actor stores no usernames or profiles, only story metadata and comment text.

**How far back can I search?** Algolia search covers the whole history of HN, back to 2007.

**Do I need an AI key?** No. Without a key the Actor runs as a fast, plain scraper and you pay only for results. To turn on AI, paste your own OpenAI or OpenRouter key into the **LLM API key** field. It is stored as an encrypted secret input and only used to call the provider you picked. LLM tokens are billed by your provider, not by us.

**Which models work?** The default is `gpt-4o-mini`, or `openai/gpt-4o-mini` on OpenRouter. Any chat model that supports JSON output works. Set it in **LLM model**.

**Can I schedule it?** Yes. Use Apify Schedules, for example daily `front_page` with `minPoints: 100`, and send the results to Slack, email or Google Sheets.

**How do I get help or request a feature?** Open an issue on the Actor's **Issues** tab.

***

**Related searches:** hacker news scraper, hacker news api, ycombinator news scraper, show hn, ask hn, hn comments, brand monitoring, tech news scraper, sentiment analysis, ai summaries.

# Actor input Schema

## `mode` (type: `string`):

Which Hacker News list to scrape, or keyword search.

## `query` (type: `string`):

Keyword(s) to search for, e.g. your brand, competitor or technology. Also narrows Show HN / Ask HN / Newest.

## `sortBy` (type: `string`):

Order of search results.

## `dateFrom` (type: `string`):

Only stories created on or after this date (YYYY-MM-DD).

## `dateTo` (type: `string`):

Only stories created on or before this date.

## `minPoints` (type: `integer`):

Skip stories with fewer points.

## `minComments` (type: `integer`):

Skip stories with fewer comments.

## `maxStories` (type: `integer`):

Maximum number of stories to return.

## `includeComments` (type: `boolean`):

Add the text of the top comments (no usernames).

## `maxCommentsPerStory` (type: `integer`):

How many top-level-first comments to fetch per story.

## `enableAi` (type: `boolean`):

Summarise each story and its discussion: summary, sentiment, key points, mentioned products, category. Requires your own OpenAI or OpenRouter key.

## `llmProvider` (type: `string`):

Which API your key belongs to.

## `llmApiKey` (type: `string`):

Your OpenAI or OpenRouter API key. Stored encrypted.

## `llmModel` (type: `string`):

Defaults to gpt-4o-mini (OpenAI) or openai/gpt-4o-mini (OpenRouter).

## Actor input object example

```json
{
  "mode": "front_page",
  "query": "AI agents",
  "sortBy": "relevance",
  "minPoints": 0,
  "minComments": 0,
  "maxStories": 50,
  "includeComments": false,
  "maxCommentsPerStory": 30,
  "enableAi": false,
  "llmProvider": "openai"
}
```

# Actor output Schema

## `stories` (type: `string`):

No description

## `aiDigest` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "query": "AI agents"
};

// Run the Actor and wait for it to finish
const run = await client.actor("data-drip/hacker-news-ai-digest").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "query": "AI agents" }

# Run the Actor and wait for it to finish
run = client.actor("data-drip/hacker-news-ai-digest").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "query": "AI agents"
}' |
apify call data-drip/hacker-news-ai-digest --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,data-drip/hacker-news-ai-digest"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/L4S2iCCLO1zn4XGmh/builds/016KoveYNBogg8XlH/openapi.json