# Hacker News Search Scraper (`deriverge/hacker-news-search`) Actor

Hacker News stories, Ask HN, Show HN, jobs and polls that match your search terms, with points, comment count, link, date and optional comment threads. Filter by date, minimum points and comments, or schedule it as a keyword alert that returns only new stories.

- **URL**: https://apify.com/deriverge/hacker-news-search.md
- **Developed by:** [deriverge s.r.o.](https://apify.com/deriverge) (community)
- **Categories:** News, Social media, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.25 / 1,000 stories

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Hacker News Search Scraper

Hacker News Search Scraper runs full-text searches over Hacker News and returns one row per matching story, Ask HN, Show HN, job or poll: title, link and its domain, the text of text posts, the submitter's username, points, comment count, date and the story's place in the results. Type a term such as `web scraping` and click **Start**. Filters cover dates, minimum points and minimum comments, and each story's comment thread can come along. One search term returns at most 1,000 stories, so long periods need splitting by date. It costs $0.50 per 1,000 stories on the Free plan ($0.25 on Business), and the $5 of monthly credit in Apify's Free plan covers about 10,000 stories.

### What data does it return?

| Field | Description |
|---|---|
| `id`, `hnUrl` | Hacker News item ID and its discussion page, such as `https://news.ycombinator.com/item?id=31075396`. |
| `title` | Title as submitted, with prefixes such as `Ask HN:` or `Show HN:` kept. |
| `kind` | `story` for a link, `ask` for Ask HN and Tell HN, `show` for Show HN, `job` or `poll`. |
| `url`, `domain` | The page the story links to and its host without `www.`, such as `techcrunch.com`. Both are `null` for text posts. |
| `text` | Body of a text post (Ask HN, a Show HN without a link, a job ad) as plain text, `null` for link stories. 9 of the 100 stories for `web scraping` had one. |
| `author` | Hacker News username of the submitter. |
| `points`, `commentsCount` | Score and number of comments when the search ran. |
| `createdAt` | When the story was posted, ISO 8601 in UTC. |
| `rank`, `matchedQuery` | Position in the results of the search term, `1` first, and the term itself. `matchedQuery` is `null` for a search with filters only. |
| `type`, `source`, `key` | `story`, `search`, and the key `story:<id>` that new-only mode compares between runs. |

With **Include comments** on, each story is followed by its comments as rows of `type` `comment`, with `storyId`, `storyTitle`, `parentId` (`null` for a top-level comment), `depth` (`0` at the top), `author`, `createdAt`, `text` and `hnUrl`. The story row then also has `commentsRead`, the number of its comments returned, while `commentsCount` stays the total of the thread.

### How to search Hacker News

1. In **Search terms**, add one term per line, such as `postgres` or `web scraping`. The search looks at titles, links and the text of text posts. An empty term together with a filter returns everything that matches the filter, for example every Show HN since a date, always newest first.
2. Under **What to search**, pick **All stories**, **Ask HN**, **Show HN**, **Jobs**, **Polls** or **Front page stories**, which are stories that reached the front page.
3. Choose the **Order**: **Relevance** ranks by match and popularity, **Newest first** by date. Narrow the results with **Published after** and **Published before** (`YYYY-MM-DD`), **Minimum points** and **Minimum comments**.
4. Set **Maximum results per search term** (100 by default). **Keywords (any of)** keeps only stories whose title, text or link contains one of the words.
5. Turn on **Include comments** for the threads, capped by **Maximum comments per story** (100 by default), and click **Start**.
6. The rows appear in the **Output** tab, ready to export as CSV, Excel or JSON. The log gives the number of matches Hacker News holds for each term.

From the API, this input asks for well-scored stories about two databases since the start of 2026, newest first:

```json
{
  "queries": ["postgres", "sqlite"],
  "searchTags": "story",
  "sortBy": "date",
  "publishedAfter": "2026-01-01",
  "minPoints": 50,
  "maxResultsPerQuery": 500
}
```

### Example output

The second result for `web scraping` by relevance in a run on 11 October 2026. The submitter's username is replaced with `hn_user`; everything else is as returned.

```json
{
  "type": "story",
  "key": "story:31075396",
  "id": 31075396,
  "kind": "story",
  "title": "Web scraping is legal, US appeals court reaffirms",
  "url": "https://techcrunch.com/2022/04/18/web-scraping-legal-court/",
  "domain": "techcrunch.com",
  "text": null,
  "author": "hn_user",
  "points": 1040,
  "commentsCount": 237,
  "createdAt": "2022-04-18T19:37:55.000Z",
  "hnUrl": "https://news.ycombinator.com/item?id=31075396",
  "source": "search",
  "rank": 2,
  "matchedQuery": "web scraping",
  "commentsRead": null
}
```

### How much does it cost to search Hacker News?

| | Free plan | Starter | Scale | Business |
|---|---|---|---|---|
| 1,000 stories | $0.50 | $0.40 | $0.325 | $0.25 |
| Comment, per 1,000 | $0.20 | $0.16 | $0.13 | $0.10 |

You pay only for the events in the table. There is no start fee, and compute time and proxies are included.

The first 1,000 stories for a term cost $0.50 on the Free plan and $0.25 on Business. A search for Show HN posts about `web scraping` since 2025 with at least 10 points found 37 stories, and with up to 50 comments each it returned 608 comments: $0.0185 plus $0.1216 on the Free plan. Stories that **Keywords (any of)** removes and rows a watch has already returned in new-only mode are not charged.

### Limits

- Hacker News search returns at most 1,000 results per query. On 11 October 2026 `llm` had 53,946 matching stories; a run with a higher maximum returned the first 1,000 by relevance, posted between January 2023 and October 2026. To get more, run the term with several **Published after** and **Published before** ranges or a higher **Minimum points**.
- The search does not cover comments on their own: comments arrive only as the threads of the stories found.
- Points and comment counts change while a story is on the front page. Each row holds the values from the moment of the run.
- Deleted comments are left out. Replies to a deleted comment keep its ID in `parentId`, so a few threads have a parent that is not in the dataset.
- Text arrives as plain text: paragraphs are kept, while links and formatting are reduced to their visible text.
- A run is fast: 1,000 stories took 7 seconds, and 37 stories with 608 comments 9 seconds, with 4 threads read at a time.

### Keyword alerts

Set **Order** to **Newest first**, turn on **Return only rows new since the last run**, give a **Watch name** or save the input as a task, and schedule it, for example every hour. Each run returns only the stories that no earlier run of the watch returned, and only those are charged. With **Relevance** order an older story that rises into the top results also counts as new, so alerts work best by date. Stories that **Maximum rows** or your spending limit cut off are not recorded as seen, so the next run returns them. Every run with a watch also writes a `CHANGES` record with the stories that appeared, the ones that dropped out of the results and the stories whose points changed.

### FAQ

#### Is it legal to scrape Hacker News?

The actor reads the public Hacker News search API at hn.algolia.com, the search linked from news.ycombinator.com, without logging in. Stories, comments and usernames are posted publicly; the actor does not collect user profiles, karma, email addresses or anything behind an account. You are responsible for how you use the data.

#### How do I get every story about a topic?

Split the period. For a busy term, run it once per year or per month with **Published after** and **Published before**, so that each range holds fewer than 1,000 stories. Rows from several runs merge by `id`.

#### Can I get whole comment threads for stories I already have?

Yes. Hacker News Comments Scraper takes story links or IDs and returns their full threads, up to 1,000 comments per story by default.

### Related scrapers

- [Substack Scraper](https://apify.com/deriverge/substack-scraper)

### Support

This actor is built and maintained by deriverge s.r.o., a software company based in the Czech Republic. If a run fails or a field you need is missing, please open an issue in the **Issues** tab or write to us at info@deriverge.com. We respond in English and Czech. Runs can be scheduled in Apify Console or started from the **API** tab, which has examples for Python, JavaScript and cURL and works with Make, Zapier, n8n and the Apify MCP server. If the actor saves you time, a short review helps other people find it.

# Changelog

This Actor's version history is a separate document: https://apify.com/deriverge/hacker-news-search/changelog.md

# Actor input Schema

## `queries` (type: `array`):

Full-text search over titles, links and text. An empty term with a tag or date filter returns everything that matches the filter.

## `searchTags` (type: `string`):

Ask HN and Show HN are subsets of stories; front page means stories that reached the front page.

## `sortBy` (type: `string`):

Relevance ranks by match and popularity; newest first is what a monitor wants. An empty search term always uses newest first.

## `publishedAfter` (type: `string`):

Only items created after this date (YYYY-MM-DD).

## `publishedBefore` (type: `string`):

Only items created before this date (YYYY-MM-DD).

## `minPoints` (type: `integer`):

Only items with at least this many points.

## `minComments` (type: `integer`):

Only items with at least this many comments.

## `maxResultsPerQuery` (type: `integer`):

Results are read a hundred at a time from the top of the ranking.

## `includeComments` (type: `boolean`):

Return the comment tree of every story as rows with author, date, text, depth and parent. Charged per comment.

## `maxCommentsPerStory` (type: `integer`):

Replies count towards the cap; the tree is walked from the top.

## `keywords` (type: `array`):

Keep only stories whose title, text or link contain at least one of these words, case-insensitive. Dropped stories are never charged.

## `newOnly` (type: `boolean`):

Keeps a snapshot per watch name (or per saved task) and returns only stories that were not there before. Schedule it hourly and you have a keyword alert for Hacker News that bills only for new stories.

## `watchName` (type: `string`):

Name of the snapshot used by the new-only mode, for example "ai-alert". Runs from a saved task get a snapshot automatically even without a name.

## `maxItems` (type: `integer`):

Hard cap on returned rows in the run.

## Actor input object example

```json
{
  "queries": [
    "web scraping"
  ],
  "searchTags": "story",
  "sortBy": "relevance",
  "maxResultsPerQuery": 100,
  "includeComments": false,
  "maxCommentsPerStory": 100,
  "newOnly": false,
  "maxItems": 10000
}
```

# Actor output Schema

## `rows` (type: `string`):

One row per matching story with the search term, rank, title, link, points, comments, author and date.

## `changes` (type: `string`):

Stories that appeared since the previous snapshot of the same watch name or task.

## `summary` (type: `string`):

Counts per list and search term, what filters dropped, what was charged and notes.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "web scraping"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("deriverge/hacker-news-search").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "queries": ["web scraping"] }

# Run the Actor and wait for it to finish
run = client.actor("deriverge/hacker-news-search").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "web scraping"
  ]
}' |
apify call deriverge/hacker-news-search --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,deriverge/hacker-news-search"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/dwI7t1NU85jd9qhor/builds/mXdjEN0RjkkgWhwv3/openapi.json