# Hacker News Scraper: Stories, Comments, Search, Jobs (`fernandosord/hacker-news-scraper`) Actor

Scrape Hacker News: the live front page, newest, best, Ask HN, Show HN and jobs lists, keyword search with dates and minimum points, full comment threads, any user's submissions and the monthly Who is hiring thread. One item per story or comment with points, comments, author, date and plain text.

- **URL**: https://apify.com/fernandosord/hacker-news-scraper.md
- **Developed by:** [Plainsource](https://apify.com/fernandosord) (community)
- **Categories:** News, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.50 / 1,000 story or comments

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

This Hacker News scraper returns stories and comments from Hacker News as clean rows: the live front page and the newest, best, Ask HN, Show HN and jobs lists, keyword search across all of Hacker News since 2006, full comment threads, any user's submissions, and the monthly "Who is hiring?" thread. It uses the official Hacker News API and Algolia's HN Search API, so runs are fast, cheap and need no login or API key.

- **Live lists:** the front page in its current order, with rank, points and comment count read at run time.
- **Search:** keywords with a date range, a minimum points value and relevance or newest-first order. Newest-first searches go past Algolia's 1,000-result limit.
- **Comments:** every comment under a story with author, depth and parent, as separate rows you can sort and filter.
- **Who is hiring:** one row per job post from any month's thread, with the post's first line (usually Company | Role | Location) and a remote flag. Filter by words such as `rust remote`.

### What data you get

One item per story or comment:

| Field | What it holds |
|---|---|
| `id`, `hnUrl` | Hacker News item ID and link |
| `type`, `category` | story, comment, job or poll; category also tells Ask HN, Show HN, Launch HN and Tell HN apart |
| `title`, `url`, `domain` | Story title, the link it points to and that link's website |
| `author` | Username |
| `points`, `numComments` | Story points and total comments. For a comment, `numComments` is the number of replies under it |
| `createdAt`, `createdAtUnix` | When it was posted (UTC) |
| `text` | Comment text, or the body of an Ask HN, Show HN or job post, as plain text with full link URLs |
| `parentId`, `storyId`, `storyTitle`, `depth` | Where a comment sits: what it replies to, the story it belongs to and how deep it is nested |
| `rank` | Position on the live list at run time (list modes) |
| `hiringHeadline`, `isRemote` | First line of a job post and whether it mentions remote (Who is hiring mode) |
| `source` | The list, search, user, story or thread the item came from |

### How to use it

1. Pick **What to scrape**.
2. Fill the field for that mode: nothing for live lists, a **Search query** for search, **Story IDs or URLs** for comments, **Usernames** for user mode, or a **Month** for Who is hiring.
3. Optional: set **From date**, **To date**, **Minimum points**, and turn on **Include comments** to get each story's comments after it.
4. Run it and download JSON, CSV or Excel, or call it from the API, Make, Zapier, n8n or an AI agent through the Apify MCP server.

#### Example input

```json
{
  "mode": "search",
  "query": "rust",
  "dateFrom": "2024-01-01",
  "dateTo": "2025-12-31",
  "minPoints": 300,
  "maxItems": 5,
  "includeComments": true,
  "maxCommentsPerStory": 3
}
```

#### Example output (from real runs)

A story from the front page:

```json
{
  "id": 50289418,
  "type": "story",
  "category": "story",
  "title": "We're releasing the weights of Mistral Large 4",
  "url": "https://mistral.ai/news/mistral-large-4-oss/",
  "domain": "mistral.ai",
  "author": "kuberwastaken",
  "points": 1219,
  "numComments": 341,
  "createdAt": "2026-10-29T14:13:21.000Z",
  "createdAtUnix": 1793283201,
  "text": null,
  "storyId": 50289418,
  "rank": 2,
  "hnUrl": "https://news.ycombinator.com/item?id=50289418",
  "source": "front page"
}
```

A comment collected with **Include comments**:

```json
{
  "id": 40172273,
  "type": "comment",
  "author": "maximilianburke",
  "numComments": 3,
  "createdAt": "2024-04-26T17:55:51.000Z",
  "text": "I would love to be able to bypass the orphan rule for internal crates.",
  "parentId": 40172033,
  "storyId": 40172033,
  "storyTitle": "Leaving Rust gamedev after 3 years",
  "depth": 1,
  "hnUrl": "https://news.ycombinator.com/item?id=40172273",
  "source": "search: rust"
}
```

A job post from the Who is hiring thread (text shortened):

```json
{
  "id": 49926618,
  "type": "comment",
  "author": "futohq",
  "createdAt": "2026-10-01T20:22:29.000Z",
  "text": "FUTO | https://futo.tech/ | Austin, TX | REMOTE(Worldwide) or ONSITE | Full Time\n\nFUTO is an organization dedicated to developing ...",
  "parentId": 49922569,
  "storyTitle": "Ask HN: Who is hiring? (October 2026)",
  "hiringHeadline": "FUTO | https://futo.tech/ | Austin, TX | REMOTE(Worldwide) or ONSITE | Full Time",
  "isRemote": true,
  "hnUrl": "https://news.ycombinator.com/item?id=49926618"
}
```

### How much does it cost?

You pay per item saved: **$0.0005 per story, comment or job post**, plus **$0.0005 once per run**. 1,000 items cost $0.50 plus the start fee, so the front page (30 stories) costs about $0.02. Apify's free plan credit of $5 a month covers about 10,000 items.

Every comment is its own item, so a story with 500 comments costs $0.25 when you collect all of them. Inputs that fail (an unknown user, a missing item ID) cost nothing and are listed with the reason in the `SUMMARY` record of the run's key-value store.

### Use cases

- **Daily digest:** schedule the front page or Best list with **Minimum points** set to 100 and send it to Slack, email or a spreadsheet.
- **Brand and launch monitoring:** search your product or company name newest first with **From date** set to `1 day`.
- **Job market research:** pull the Who is hiring thread every month and filter by `remote`, a language or a city.
- **Research and AI datasets:** collect comment threads on a topic as plain text for sentiment analysis, summaries or RAG.

### FAQ

**Are points and comment counts live?**
In the live list modes and when a story's comments are fetched, yes: they come from the official Hacker News API at run time. In search and user modes they come from Algolia's index, which can trail the live site by a few minutes.

**Why do comments have no points?**
Hacker News does not publish comment scores anywhere, so no tool can get them.

**How far back can it search?**
Back to 2006, the first months of Hacker News. Relevance order stops at 1,000 results per search, a limit of Algolia's API. Newest-first order keeps going by moving the date window back, so it can return far more.

**In what order are comments?**
Top-level comments follow Hacker News's own ranking. Replies under each comment are in the order they were posted, nested right under their parent. Deleted comments are left out.

**What does Max results count?**
Stories for live lists and search, items per user in user mode, and job posts in Who is hiring mode. Comments mode uses **Max comments per story**. Comments added with **Include comments** do not count toward Max results, so set **Max comments per story** to control cost.

**Does the Freelancer thread work?**
Yes for months up to October 2025, which is the last Freelancer thread the whoishiring account posted. The Who is hiring and Who wants to be hired threads work for every month.

**Is this legal?**
It reads public data through the official Hacker News API and Algolia's public HN Search API. You are responsible for how you use it, especially personal data, under laws like GDPR.

### Other Actors by this developer

- [Google News Scraper](https://apify.com/fernandosord/google-news-rss-scraper): headlines by keyword, topic or country with the real publisher URL.
- [Substack Scraper](https://apify.com/fernandosord/substack-scraper-posts): posts, likes, comments and full text from any Substack newsletter.
- [Telegram Channel Scraper](https://apify.com/fernandosord/telegram-channel-scraper): posts, views and reactions from public Telegram channels, no login.
- [Career Site Jobs API](https://apify.com/fernandosord/career-site-jobs): open jobs from Greenhouse, Lever, Ashby, Workable and Recruitee career sites.
- [Website Tech Stack Detector](https://apify.com/fernandosord/website-tech-stack-detector): see which frameworks and tools a website runs on.

# Actor input Schema

## `mode` (type: `string`):

A live list (front page, newest, best, Ask HN, Show HN, jobs), a keyword search over all of Hacker News, the comments of given stories, the submissions of given users, or a monthly Who is hiring thread.

## `query` (type: `string`):

Search mode: keywords to find in titles, URLs, story text and comments. User mode: only keep that user's items matching these words. Who is hiring mode: only keep job posts that contain all of these words (for example: rust remote).

## `contentType` (type: `string`):

What to return in Search and User modes.

## `searchSort` (type: `string`):

Relevance ranks by match quality and points, at most 1,000 results per search. Date returns newest first and can go past 1,000 results.

## `storyIds` (type: `array`):

Story IDs (8863) or links (https://news.ycombinator.com/item?id=8863). The Actor returns every comment, nested replies included, up to Max comments per story.

## `usernames` (type: `array`):

Hacker News usernames (pg, dang) or profile links (https://news.ycombinator.com/user?id=pg). Usernames are case-sensitive. Newest submissions first.

## `hiringMonth` (type: `string`):

latest for the current thread, or a month such as 2026-10 or October 2026.

## `hiringThread` (type: `string`):

Which of the monthly threads to read. Each top-level comment becomes one item. The Freelancer thread ran until October 2025.

## `dateFrom` (type: `string`):

Optional. Only items posted on or after this date (2026-01-31) or within a relative period (7 days, 3 months).

## `dateTo` (type: `string`):

Optional. Only items posted on or before this date. A bare date includes the whole day (UTC).

## `minPoints` (type: `integer`):

Skip stories with fewer points. Comments have no public points and are not affected.

## `maxItems` (type: `integer`):

Stories per live list or search, items per user in user mode, job posts in Who is hiring mode. Comments mode uses Max comments per story instead, and comments added with Include comments do not count here. 0 means no cap.

## `includeComments` (type: `boolean`):

List, search and user modes: after each story, add its comments as separate items (one extra request per story). Every comment is an item.

## `maxCommentsPerStory` (type: `integer`):

Cap on comments saved for each story, nested replies included, in comments mode and with Include comments. Top-level comments follow Hacker News's own ranking, replies are in time order.

## Actor input object example

```json
{
  "mode": "front",
  "contentType": "story",
  "searchSort": "relevance",
  "hiringMonth": "latest",
  "hiringThread": "hiring",
  "minPoints": 0,
  "maxItems": 30,
  "includeComments": false,
  "maxCommentsPerStory": 100
}
```

# Actor output Schema

## `results` (type: `string`):

One item per story or comment, in list order, search order or thread order.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "mode": "front",
    "maxItems": 30
};

// Run the Actor and wait for it to finish
const run = await client.actor("fernandosord/hacker-news-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "mode": "front",
    "maxItems": 30,
}

# Run the Actor and wait for it to finish
run = client.actor("fernandosord/hacker-news-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "mode": "front",
  "maxItems": 30
}' |
apify call fernandosord/hacker-news-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,fernandosord/hacker-news-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/MeeJh61b0696WUStV/builds/lrgY38ZECTupun1en/openapi.json