# Hacker News Scraper: Front Page, Search, Comments & Jobs (`manageable_gully/hacker-news-scraper`) Actor

Scrape Hacker News through the official HN API and HN Search API: front page/top, new, best, Ask HN, Show HN and jobs, full-text search of stories and comments since 2006, comment threads, points, authors, domains. Keyword alerts with only-new mode. $0.20 per 1,000 items.

- **URL**: https://apify.com/manageable_gully/hacker-news-scraper.md
- **Developed by:** [WebIntel Tools](https://apify.com/manageable_gully) (community)
- **Categories:** News, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.20 / 1,000 hacker news items

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Hacker News Scraper: Front Page, Search, Comments, Ask HN, Show HN & Jobs

Get **Hacker News data as clean JSON, CSV or Excel** through the **official Hacker News API** and the **HN Search API (Algolia)**: the live front page (top), new, best, Ask HN, Show HN and job lists, **full-text search of every story and comment since 2006**, and complete **comment threads**. Each row has the title, link, domain, points, comment count, author, time, text and the HN discussion link.

No HTML scraping, no login, no proxy: fast, cheap and reliable. **$0.20 per 1,000 items.**

### What you can use it for

- **Brand and competitor monitoring:** schedule a search for your product or company name with *Only new items* and get every new mention (stories and comments).
- **Launch tracking:** every Show HN and Launch HN post with points and comments, to see what developers build and what gets traction.
- **Content and newsletter curation:** today's front page or the best stories of the week with points above a threshold.
- **Developer sentiment and research:** full comment threads of a discussion for analysis or an LLM summary.
- **Hiring and job research:** HN job posts (YC startups) and the monthly "Who is hiring?" threads with all comments.
- **Domain tracking:** every HN submission linking to your site (or github.com, a news outlet, a competitor).
- **AI agents (MCP) and RAG:** structured HN data an agent can query on demand.

### How to use it

1. Choose a **mode**: `list` (front page and other live lists), `search` (full-text search with filters) or `items` (specific IDs/URLs).
2. Add filters: date range, minimum points/comments, domains, authors.
3. Optional: turn on **Include comment threads** (charged per comment).
4. Click **Start**, or schedule it with **Only new items** for alerts.

### Why this scraper

- **Official APIs only:** the Firebase-hosted Hacker News API and HN Search by Algolia, both public and free. Nothing breaks when the HN page layout changes.
- **Search all of HN history:** stories, comments, Ask HN, Show HN, jobs and polls since 2006, by relevance or newest first. Newest-first search pages past HN Search's 1,000-result cap.
- **Front page with rank:** the `top` list in its live front-page order, plus `new`, `best`, `ask`, `show` and `job`.
- **Comment threads:** in thread order with depth and parent ID, limited by count and depth.
- **Only new items:** for scheduled keyword alerts you pay only for items you haven't seen; with newest-first search, a run stops as soon as it reaches items from the previous run.
- **Cheap:** $0.20 per 1,000 items, comments $0.05 per 1,000. Filtered-out, already-seen and not-found items are free. 256 MB memory.
- **Privacy-minded:** only public HN data. Usernames can be switched off with *Include usernames*.

### Modes

| Mode | What you get |
|---|---|
| `list` (default) | Live lists: `top` (front page ranking, up to 500), `new`, `best`, `ask`, `show`, `job`, with `list` and `rank`. |
| `search` | Full-text search over all HN items. `searchType`: story, comment, ask, show, job, poll, front_page or any. `sortBy`: relevance or date. |
| `items` | Specific item IDs (`8863`) or URLs (`https://news.ycombinator.com/item?id=8863`): stories, comments, jobs, polls. |

### Input examples

Today's front page, top 30 stories:

```json
{ "mode": "list", "lists": ["top"], "maxItems": 30 }
```

Every new mention of your brand (stories and comments), as an alert on a schedule:

```json
{ "mode": "search", "searchQueries": ["apify"], "searchType": "any", "sortBy": "date", "onlyNew": true, "stateKey": "brand-alert" }
```

Show HN posts from the last 7 days with 50+ points, with top-level comments:

```json
{ "mode": "search", "searchQueries": [""], "searchType": "show", "dateFrom": "7 days", "minPoints": 50, "includeComments": true, "maxCommentDepth": 1 }
```

Every HN story linking to github.com this month:

```json
{ "mode": "search", "searchType": "story", "domains": ["github.com"], "dateFrom": "2026-10-01", "sortBy": "date", "maxItems": 1000 }
```

A full discussion thread:

```json
{ "mode": "items", "itemIds": ["https://news.ycombinator.com/item?id=8863"], "includeComments": true, "maxCommentsPerStory": 0 }
```

### Output

```json
{
  "id": 8863, "type": "story", "title": "My YC app: Dropbox - Throw away your USB drive",
  "url": "http://www.getdropbox.com/u/2/screencast.html", "domain": "getdropbox.com",
  "hnUrl": "https://news.ycombinator.com/item?id=8863", "author": "dhouston",
  "points": 104, "commentsCount": 71, "createdAt": "2007-04-04T19:16:40Z", "text": null,
  "list": "top", "rank": 1, "matchedQuery": null,
  "comments": [{ "id": 8865, "parentId": 8863, "depth": 1, "author": "dhouston", "createdAt": "2007-04-04T19:22:55Z",
                 "text": "oh, and a mac port is coming :)", "hnUrl": "https://news.ycombinator.com/item?id=8865", "storyId": 8863 }],
  "commentsIncluded": 1, "scrapedAt": "2026-10-22T07:30:00Z"
}
```

`type` is one of story, ask, show, launch, job, poll or comment. Comment search hits carry `storyId`, `parentId` and the story title. A summary (items saved, already seen, filtered, total hits, API requests) is saved as `OUTPUT` in the run's key-value store.

### Pricing

Pay per event, no monthly rent:

- **$0.0002 per item** saved (**$0.20 per 1,000**)
- **$0.00005 per comment** in a thread (**$0.05 per 1,000**), only with *Include comment threads*
- $0.0003 per run start

| Example | Cost |
|---|---|
| Front page, top 30 stories | $0.0063 |
| 1,000 search results | $0.20 |
| 100 stories with 100 comments each | $0.52 |
| Daily brand alert, ~20 new mentions a day, for 30 days | $0.13 |

Filtered-out, already-seen and not-found items are free. You can set a **maximum cost per run** in the run options; the Actor stops cleanly when it is reached.

### API, integrations and MCP

- **API:** run it from any language with the Apify API or the `apify-client` for JavaScript/Python, and read results from the dataset.
- **No-code:** use the Apify app in **n8n, Make or Zapier**, or send results to Google Sheets, Slack or a webhook.
- **AI agents (MCP):** add `https://mcp.apify.com?tools=manageable_gully/hacker-news-scraper` to Claude, Cursor, VS Code or any MCP client.
- **Schedules:** run it hourly or daily from Apify Console, with *Only new items* for alerts.

### FAQ

**Is it allowed?** Yes. It uses the official Hacker News API (published by Y Combinator for public use) and the public HN Search API by Algolia, at modest request rates, without logging in. This Actor is not affiliated with Y Combinator or Algolia.

**How fresh is the data?** Lists come live from the official API. HN Search indexes new items within minutes.

**Why do points differ slightly from the website?** Points and comment counts change constantly; each row reflects the moment it was fetched.

**Are deleted or dead items included?** No. They are skipped (and not charged).

**Can I get user profiles or karma?** No. The Actor returns public usernames on items only, and you can switch them off.

### More tools from WebIntel Tools

- [RSS Feed Reader & Monitor](https://apify.com/manageable_gully/rss-feed-reader): read and monitor RSS, Atom and JSON feeds, only new items
- [Product Hunt Scraper](https://apify.com/manageable_gully/product-hunt-scraper): today's Product Hunt launches and leaderboards with websites
- [Web Page to Markdown](https://apify.com/manageable_gully/page-to-markdown): clean Markdown and RAG chunks from any URL, for LLMs
- [arXiv & PubMed Paper Search](https://apify.com/manageable_gully/arxiv-pubmed-paper-search): research papers through the official APIs

# Actor input Schema

## `mode` (type: `string`):

list = live HN lists (top/front page, new, best, Ask HN, Show HN, jobs) in their current order; search = full-text search of all HN history with filters; items = specific item IDs or news.ycombinator.com/item?id=... URLs.

## `lists` (type: `array`):

Used in 'list' mode: top (front page ranking), new, best, ask, show, job. Each list holds up to 500 items (200 for ask/show/job).

## `searchQueries` (type: `array`):

Used in 'search' mode. Full-text search (titles, URLs, text, comments) across all of Hacker News since 2006. One query per line, e.g. postgres, "open source LLM", your brand name.

## `searchType` (type: `string`):

Which items to search.

## `sortBy` (type: `string`):

relevance (HN Search ranking, up to 1,000 results per query) or date (newest first, no 1,000 cap).

## `dateFrom` (type: `string`):

Only items created on/after this date (YYYY-MM-DD) or relative, e.g. '7 days', '24 hours'. Works in all modes.

## `dateTo` (type: `string`):

Only items created before the end of this date.

## `minPoints` (type: `integer`):

Skip stories with fewer points (filtered-out items are free).

## `minComments` (type: `integer`):

Skip stories with fewer comments.

## `domains` (type: `array`):

Keep only stories linking to these domains (subdomains included), e.g. github.com, nytimes.com. Track mentions of your own site.

## `authors` (type: `array`):

Keep only items posted by these HN usernames.

## `itemIds` (type: `array`):

Used in 'items' mode: HN item IDs (8863) or URLs (https://news.ycombinator.com/item?id=8863). Stories, comments, jobs and polls.

## `includeComments` (type: `boolean`):

Adds each story's comments (thread order, with depth and parent) as a 'comments' array. Charged per comment.

## `maxCommentsPerStory` (type: `integer`):

0 = all comments.

## `maxCommentDepth` (type: `integer`):

1 = top-level comments only. 0 = no limit.

## `maxItems` (type: `integer`):

Hard cap on dataset rows (and cost). 0 = no cap besides your run's max charge.

## `includeText` (type: `boolean`):

Post / comment text as plain text (Ask HN body, job description, comment).

## `includeAuthors` (type: `boolean`):

Public HN usernames of posters and commenters. Turn off to keep usernames out of your dataset.

## `onlyNew` (type: `boolean`):

Skips items returned by a previous run with the same state key (free). Great for keyword alerts and daily digests on a schedule.

## `stateKey` (type: `string`):

Name of the saved state for 'Only new items'. Use one key per schedule.

## Actor input object example

```json
{
  "mode": "list",
  "lists": [
    "top"
  ],
  "searchQueries": [
    "open source"
  ],
  "searchType": "story",
  "sortBy": "relevance",
  "minPoints": 0,
  "minComments": 0,
  "includeComments": false,
  "maxCommentsPerStory": 100,
  "maxCommentDepth": 0,
  "maxItems": 100,
  "includeText": true,
  "includeAuthors": true,
  "onlyNew": false,
  "stateKey": "default"
}
```

# Actor output Schema

## `results` (type: `string`):

One row per story, Ask/Show HN post, job, poll or comment: title, link, domain, points, comments count, author, time, text and (optional) the comment thread.

## `summary` (type: `string`):

Items and comments saved, already-seen items skipped, filtered-out items, total search hits and API requests made.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "lists": [
        "top"
    ],
    "searchQueries": [
        "open source"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("manageable_gully/hacker-news-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "lists": ["top"],
    "searchQueries": ["open source"],
}

# Run the Actor and wait for it to finish
run = client.actor("manageable_gully/hacker-news-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "lists": [
    "top"
  ],
  "searchQueries": [
    "open source"
  ]
}' |
apify call manageable_gully/hacker-news-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,manageable_gully/hacker-news-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/fExxbs6iNvElHSdmJ/builds/44BuRadC5F3OxhvSG/openapi.json