# Hacker News Scraper: Who Is Hiring Jobs, Stories, Comments (`ctriolab/hackernews-scraper`) Actor

Scrape Hacker News stories, full comment trees, user profiles and the monthly Who Is Hiring threads as HN jobs with company, role, location, remote, salary and visa fields. Search by keyword, domain or author with date and points filters, past the 1,000-result limit. Official HN API.

- **URL**: https://apify.com/ctriolab/hackernews-scraper.md
- **Developed by:** [Ctrio Lab](https://apify.com/ctriolab) (community)
- **Categories:** Jobs, News
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.35 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Hacker News Scraper: Stories, Comments, Who's Hiring, Search

Get **Hacker News stories, full comment trees, user profiles, search results and the monthly "Who is hiring?" job posts** as clean JSON, CSV or Excel. No login, no API key, no browser. Built on the **official Hacker News API** (Firebase) and the **HN Search API** by Algolia.

**Use it for:** tech job leads from Who is hiring, hiring-market research (remote share, salary ranges, stacks), brand and launch monitoring (who links to your domain), competitor and trend research, Show HN and Ask HN analysis, sentiment analysis of big discussions, and LLM / RAG datasets of HN threads.

### What you get

| Mode | One row per | Main fields |
|---|---|---|
| **Story lists** (top, new, best, Ask HN, Show HN, jobs) | story (+ optional comments) | rank, title, URL, domain, points, comment count, author, date, text, HN link |
| **Search** (keyword, linked domain or URL, author, date range, min points, min comments) | story or comment | same as above, plus story title and parent for comments |
| **Stories by ID or URL** | story + every comment | depth, parentId, path (story/parent/.../comment), reply count, clean text, links |
| **Who is hiring / Who wants to be hired / Freelancer** | top-level post | company, role, location, remote (yes / no / hybrid), salary text, visa (yes / no), employment type, company URL, full text, `hasEmail` flag |
| **User profiles** | user | karma, join date, about text, links in about, number of submissions |

All text is converted from HN's HTML to plain text. Links are kept as a separate `links` list.

### Features

- **Search beyond 1,000 results.** The HN Search API returns at most 1,000 hits per query. In date order the Actor walks back in time automatically, so a query like `rust` from 2025-01-01 returns every match (tested: 2,500 results in 5 requests).
- **Complete comment trees.** Every comment of a story, flattened in HN display order with `depth`, `parentId` and `path`. Tested on a 1,051-comment story: 1,051 of 1,051 comments (100% of HN's own count).
- **Parsed Who is hiring posts.** Best-effort extraction of company, role, location, remote, salary, visa and employment type from the usual `Company | Role | Location | Remote | Salary` first line. Filter by keywords, remote and location. "Who wants to be hired" posts give location, remote, relocation, technologies and resume link.
- **Domain and URL monitoring.** Find every HN story linking to `yourcompany.com` or `github.com/your-org`. Fuzzy matches are removed, only real links are returned.
- **New items only mode** for daily or hourly schedules: the Actor remembers what it already returned per list, search, story or thread and outputs only new items. Runs with nothing new succeed and cost almost nothing.
- **Polite and fast fail.** Official APIs only, at most 10 parallel requests to the HN API and at most 2.5 requests per second to HN Search. Rate limits and server errors are retried with backoff. Invalid input stops the run within seconds with a clear message and no charge.

### Input examples

**Front page right now (default)**

```json
{ "mode": "lists", "lists": ["top"], "maxItems": 30 }
```

**This month's Who is hiring, remote jobs that mention Python**

```json
{
  "mode": "hiring",
  "hiringThreads": ["who_is_hiring"],
  "hiringMonths": ["latest"],
  "hiringKeywords": ["python"],
  "hiringRemote": "remote",
  "maxItems": 0
}
```

**Last 3 months of Who is hiring in Europe**

```json
{ "mode": "hiring", "hiringMonths": ["last 3"], "hiringLocations": ["Europe", "London", "Berlin", "Amsterdam"], "maxItems": 0 }
```

**Every story about Rust since 2025 with at least 50 points**

```json
{
  "mode": "search",
  "searchQueries": ["rust"],
  "searchType": "story",
  "searchSort": "date",
  "dateFrom": "2025-01-01",
  "minPoints": 50,
  "maxItems": 0
}
```

**Who links to my site (daily monitor)**

```json
{
  "mode": "search",
  "searchDomains": ["github.com/anthropics"],
  "searchType": "story",
  "onlyNewItems": true,
  "monitorName": "anthropic-github-links",
  "maxItems": 0
}
```

**Comments mentioning my product in the last 7 days**

```json
{ "mode": "search", "searchQueries": ["\"apify\""], "searchType": "comment", "dateFrom": "7 days", "maxItems": 0 }
```

**A whole discussion with every comment**

```json
{ "mode": "items", "storyIds": ["https://news.ycombinator.com/item?id=8863"] }
```

**User profiles**

```json
{ "mode": "users", "users": ["pg", "dang"] }
```

### Output example

Who is hiring row (`type: "job"`):

```json
{
  "type": "job",
  "threadType": "who_is_hiring",
  "threadTitle": "Ask HN: Who is hiring? (October 2026)",
  "threadMonth": "2026-10",
  "id": 49930681,
  "author": "PolinaLu",
  "firstLine": "Modash.io | Senior Product Engineer | Remote (Europe) | Full-time | €75k–110k | https://modash.io",
  "company": "Modash.io",
  "role": "Senior Product Engineer",
  "location": "Remote (Europe)",
  "remote": "yes",
  "salary": "€75k–110k",
  "visa": null,
  "employmentType": "full-time",
  "url": "https://modash.io",
  "text": "Modash.io | Senior Product Engineer | ...",
  "links": ["https://modash.io", "https://modash.io/engineering", "https://apply.workable.com/modash/j/C1507B65C3"],
  "hasEmail": false,
  "replyCount": 0,
  "createdAt": "2026-10-02T07:13:32.000Z",
  "hnUrl": "https://news.ycombinator.com/item?id=49930681"
}
```

Comment row (`type: "comment"`):

```json
{
  "type": "comment",
  "id": 47940125,
  "storyId": 47939579,
  "storyTitle": "Ghostty is leaving GitHub",
  "parentId": 47939809,
  "depth": 2,
  "path": "47939579/47939809/47940125",
  "author": "idan",
  "text": "Hi there! Longtime fan and hubber here. ...",
  "links": [],
  "hasEmail": false,
  "replyCount": 38,
  "createdAt": "2026-04-28T20:21:02.000Z",
  "hnUrl": "https://news.ycombinator.com/item?id=47940125"
}
```

A `RUN_SUMMARY` record in the default key-value store lists, per list, query, story and thread: rows saved, total matches, comment completeness versus HN's count, non-posts skipped, rows filtered out and errors.

### Email addresses

Email addresses in posts and profiles are **replaced with `[email hidden]`** and only a `hasEmail: true/false` flag is returned. This Actor is not a tool for collecting personal email addresses. Use the `hnUrl` to read the original post and contact people the way they asked to be contacted.

### Pricing

Pay per result: **$0.50 per 1,000 results** plus a tiny Actor start fee. You pay for each row saved to the dataset, nothing for requests that return nothing. Runs that find nothing new in new-only mode cost almost nothing.

### Tips

- **Who is hiring fields are best effort.** Posts are free text. `company`, `role`, `location`, `remote`, `salary` and `visa` are filled when the post follows the usual pattern. `text` always has the full post.
- **Relevance vs date order.** Relevance is limited to 1,000 results by the search API. Use date order (default) for anything bigger.
- **Points and comment counts** in search results come from the search index and can lag a few minutes. Lists and comment trees read live data from the official API.
- **Deleted and flagged (dead) comments** are not returned. Their visible replies are, with the original `parentId`.
- Usernames are case-sensitive.

### Use with AI agents (MCP)

Call this Actor from Claude, ChatGPT or any MCP client through the Apify MCP server (`https://mcp.apify.com`), for example: "Get this month's remote Who is hiring jobs that mention Rust".

### FAQ

**Is it legal to scrape Hacker News?** This Actor does not scrape the website. It reads the official Hacker News API that Y Combinator publishes for developers, and the public HN Search API. You are responsible for how you use the data, including the privacy of the people who wrote it.

**Do I need a Hacker News account or API key?** No.

**How fresh is the data?** Lists and comment trees are live. Search results are usually seconds to minutes behind.

**Can I get all comments of a user?** Yes: mode search, `searchAuthor` set to the username, `searchType` comment, `maxItems` 0.

This Actor is not affiliated with Y Combinator, Hacker News or Algolia.

# Actor input Schema

## `mode` (type: `string`):

Pick one mode. Only the fields of the chosen mode are used.

## `maxItems` (type: `integer`):

0 means no limit. Lists hold up to 500 stories (top, new), 200 (best) or fewer (ask, show, job). Search in date order can go far beyond 1,000 results. Comments are limited separately below.

## `lists` (type: `array`):

Mode: lists. One row per story with its rank in the list.

## `includeComments` (type: `boolean`):

Mode: lists. Also return the comments of each story (one row per comment with depth, parent and path). Mode items always returns comments.

## `maxCommentDepth` (type: `integer`):

1 = only top-level comments, 2 = top-level and their replies. 0 = the whole tree. Used in modes lists and items.

## `maxCommentsPerStory` (type: `integer`):

0 = all comments. Comments are fetched level by level, so a limit keeps the top of the discussion.

## `searchQueries` (type: `array`):

Mode: search. Each keyword (or phrase) is a separate search. Leave empty to search only by domain, author or dates.

## `searchDomains` (type: `array`):

Mode: search. Find stories that link to a domain (github.com) or URL prefix (github.com/anthropics). Results are checked so only real matches are returned.

## `searchAuthor` (type: `string`):

Mode: search. Only items posted by this HN username (case-sensitive).

## `searchType` (type: `string`):

Mode: search.

## `searchSort` (type: `string`):

Mode: search. Date order walks back in time and can return tens of thousands of results. Relevance order is limited to 1,000 results by the search API.

## `dateFrom` (type: `string`):

Mode: search. 2026-01-01, 2026-01-01T12:00:00Z or relative (24 hours, 7 days, 3 months).

## `dateTo` (type: `string`):

Mode: search. Inclusive when you give a day (2026-09-30 includes that whole day, UTC).

## `minPoints` (type: `integer`):

Mode: search. Stories only (the search API has no points for comments).

## `minComments` (type: `integer`):

Mode: search. Stories only.

## `storyIds` (type: `array`):

Mode: items. Numeric IDs (8863) or HN URLs. Returns the story plus every comment as a flat row with depth, parentId and path (story/parent/.../comment), in the same order as on HN. A comment ID returns that comment and its replies.

## `hiringThreads` (type: `array`):

Mode: hiring. Monthly threads by the whoishiring account. Each top-level comment becomes one row with parsed fields and the full text.

## `hiringMonths` (type: `array`):

Mode: hiring. "latest", "last 3" (the 3 newest threads) or months like 2026-09.

## `hiringKeywords` (type: `array`):

Mode: hiring. Keep posts that mention at least one keyword (case-insensitive), for example rust, python, data engineer.

## `hiringRemote` (type: `string`):

Mode: hiring. Uses the parsed remote field (best effort).

## `hiringLocations` (type: `array`):

Mode: hiring. Keep posts whose location or first line contains one of these (London, Berlin, US, Europe ...).

## `hiringSkipNonPosts` (type: `boolean`):

Mode: hiring. Skips meta comments (for example a reply about the thread itself) that do not look like a job or candidate post.

## `users` (type: `array`):

Mode: users. Case-sensitive HN usernames or profile URLs.

## `onlyNewItems` (type: `boolean`):

For schedules. Remembers returned IDs in a named key-value store per list, search, story or thread. Later runs return only new stories, comments, search results or hiring posts. Runs with nothing new succeed and cost almost nothing. User profiles are always returned.

## `monitorName` (type: `string`):

Use a different name for each schedule so their memories do not mix.

## `maxRetries` (type: `integer`):

Retries on rate limits (HTTP 429), server errors and network errors with exponential backoff.

## Actor input object example

```json
{
  "mode": "lists",
  "maxItems": 30,
  "lists": [
    "top"
  ],
  "includeComments": false,
  "maxCommentDepth": 0,
  "maxCommentsPerStory": 0,
  "searchQueries": [
    "rust"
  ],
  "searchDomains": [],
  "searchType": "story",
  "searchSort": "date",
  "minPoints": 0,
  "minComments": 0,
  "storyIds": [
    "https://news.ycombinator.com/item?id=8863"
  ],
  "hiringThreads": [
    "who_is_hiring"
  ],
  "hiringMonths": [
    "latest"
  ],
  "hiringKeywords": [],
  "hiringRemote": "any",
  "hiringLocations": [],
  "hiringSkipNonPosts": true,
  "users": [
    "pg"
  ],
  "onlyNewItems": false,
  "monitorName": "default",
  "maxRetries": 5
}
```

# Actor output Schema

## `stories` (type: `string`):

Stories, search results and comments (type = story or comment).

## `jobs` (type: `string`):

Parsed job posts from the monthly Who is hiring threads (type = job).

## `users` (type: `string`):

User profiles (type = user).

## `summary` (type: `string`):

Items, matches, completeness and errors per list, query, story and thread.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "lists": [
        "top"
    ],
    "searchQueries": [
        "rust"
    ],
    "storyIds": [
        "https://news.ycombinator.com/item?id=8863"
    ],
    "hiringThreads": [
        "who_is_hiring"
    ],
    "hiringMonths": [
        "latest"
    ],
    "users": [
        "pg"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("ctriolab/hackernews-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "lists": ["top"],
    "searchQueries": ["rust"],
    "storyIds": ["https://news.ycombinator.com/item?id=8863"],
    "hiringThreads": ["who_is_hiring"],
    "hiringMonths": ["latest"],
    "users": ["pg"],
}

# Run the Actor and wait for it to finish
run = client.actor("ctriolab/hackernews-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "lists": [
    "top"
  ],
  "searchQueries": [
    "rust"
  ],
  "storyIds": [
    "https://news.ycombinator.com/item?id=8863"
  ],
  "hiringThreads": [
    "who_is_hiring"
  ],
  "hiringMonths": [
    "latest"
  ],
  "users": [
    "pg"
  ]
}' |
apify call ctriolab/hackernews-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,ctriolab/hackernews-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/3i2uBqs3CToCQy4bZ/builds/ar7C3if0BR9CKD2ir/openapi.json