# Hacker News Scraper — stories, comments, jobs, Who is hiring (`arthursbuisness/hacker-news-stories-comments-jobs`) Actor

- **URL**: https://apify.com/arthursbuisness/hacker-news-stories-comments-jobs.md
- **Developed by:** [Arthur](https://apify.com/arthursbuisness) (community)
- **Categories:** News, Developer tools, Business
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$0.20 / 1,000 stored item (story, comment, job or candidate)s

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Hacker News Scraper — stories, comments, jobs, Who is hiring

Scrape **Hacker News** into a clean dataset: the front page, newest, best, Ask HN, Show HN and job listings; **full-text search** with date and points filters; **comment threads** (depth-limited); and the monthly **"Who is hiring?"** thread parsed into structured **job rows** (company, role, location, remote/onsite/hybrid, salary, links, contact e-mails). Uses HN's official Firebase API and the Algolia HN search API — **no login, no browser, no proxies**, fast and cheap.

Typical uses: tech-news monitoring and digests, brand/competitor/keyword alerts (e.g. every story mentioning your product), research datasets and LLM inputs, job boards and recruiting leads from "Who is hiring?", Show HN launch tracking, sentiment analysis on comments.

### What you get

**Stories** (`type: "story"`, modes top/new/best/ask/show/jobs/search):

| field | description |
|---|---|
| `id`, `rank`, `kind` | HN item id, position in the list, `story` / `ask` / `show` / `job` / `poll` |
| `title`, `url`, `domain`, `text` | Link and (for Ask/Show/Jobs) the post body as plain text |
| `author`, `points`, `num_comments`, `created_at` | Metadata (UTC ISO timestamp) |
| `hn_url` | `https://news.ycombinator.com/item?id=…` |

**Comments** (`type: "comment"`, with `includeComments` or `searchTags: comment`): `id`, `story_id`, `story_title`, `story_url`, `parent_id`, `depth`, `author`, `created_at`, `text` (plain text, HTML entities decoded), `hn_url`.

**Jobs** (`type: "job"`, mode whoishiring): `company`, `role`, `location`, `remote`, `onsite`, `hybrid`, `visa`, `interns`, `salary` (first salary range found, e.g. `$170-240K`), `urls`, `emails`, `header` (the post's first line), `header_parts` (the first line split on `|`), `text` (full plain text), `thread_title`, `month`, `author`, `created_at`, `hn_url`.

**Candidates** (`type: "candidate"`, thread "Who wants to be hired?"): `location`, `remote`, `willing_to_relocate`, `technologies`, `resume`, `email`, `header_parts`, `urls`, `emails`, `text`.

Problems (bad mode, no matching thread, API errors) produce a `type: "error"` row — free.

Export as JSON, CSV or Excel, or push rows to Google Sheets, Make, Zapier, webhooks or your code through the Apify API.

### Input

| field | default | meaning |
|---|---|---|
| `mode` | `top` | `top`, `new`, `best`, `ask`, `show`, `jobs`, `search`, `whoishiring` |
| `query` | — | search text (mode `search`) |
| `searchTags` | `story` | Algolia tags: `story`, `comment`, `poll`, `show_hn`, `ask_hn`, `front_page`, `job`, `author_pg`, `story_8863`; `,` = AND, `(a,b)` = OR |
| `sortBy` | `relevance` | `relevance` or `date` |
| `dateFrom`, `dateTo` | — | search window, `YYYY-MM-DD` (UTC) |
| `minPoints` | 0 | search: minimum upvotes |
| `maxItems` | 100 | max story/job rows (lists: up to 500; search: up to 5,000) |
| `includeComments` | false | add comment rows per story |
| `commentDepth` | 1 | 1 = top-level only, 2 = plus replies, … |
| `maxCommentsPerStory` | 100 | cap per story |
| `hiringThread` | `who-is-hiring` | `who-is-hiring`, `who-wants-to-be-hired`, `freelancer` |
| `hiringMonth` | latest | `YYYY-MM` to pick an older thread |

Examples:

```json
{ "mode": "whoishiring", "maxItems": 1000 }
```

```json
{ "mode": "search", "query": "postgres", "sortBy": "date", "dateFrom": "2026-08-01", "minPoints": 50, "maxItems": 200, "includeComments": true, "commentDepth": 1, "maxCommentsPerStory": 20 }
```

Schedule the actor (hourly for `new`/`search`, monthly for `whoishiring`) and deduplicate on `id` to build a continuous monitor.

### Pricing

Pay per event: **$0.0002 per stored row** ($0.20 per 1,000 stories, comments or jobs). Error rows are free. Apify platform usage (a few seconds of compute per run) is billed separately by Apify.

### Limitations — please read

- List modes (`top`, `new`, …) return at most the 500 items HN publishes; story metadata comes from Algolia's index (with a per-item Firebase fallback), so `points`/`num_comments` can lag HN by a few minutes.
- Comments are loaded from Algolia in one call per story (up to 5,000 per story); `depth` is computed from the parent chain. Deleted/empty comments are skipped.
- "Who is hiring?" parsing follows the community convention `Company | Role | Location | …` on the first line. Posts that ignore the convention still get `header`, `header_parts`, `text`, `urls` and `emails`, but `company`/`role`/`location` may be off. `remote`/`onsite`/`hybrid` are keyword flags, not verified facts. Candidate posts are even less uniform — treat those fields as best-effort.
- Search is limited by Algolia to 50 pages × 100 hits.

Not affiliated with Y Combinator or Hacker News. Public data only; respect people's contact preferences when using e-mails from posts.

# Actor input Schema

## `mode` (type: `string`):

Which list to scrape. 'search' uses the query and filters below; 'whoishiring' parses the latest monthly 'Ask HN: Who is hiring?' (or 'Who wants to be hired?' / 'Freelancer?') thread into one row per post.

## `query` (type: `string`):

Full-text query for mode 'search' (matches title, URL, author and text). Leave empty to list by filters only.

## `searchTags` (type: `string`):

Algolia tag filter for mode 'search': story, comment, poll, show\_hn, ask\_hn, front\_page, job, author\_<username>, story\_<id>. Combine with commas (AND) or parentheses (OR), e.g. '(story,poll)' or 'comment,author\_pg'. Default: story.

## `sortBy` (type: `string`):

'relevance' = Algolia relevance + points; 'date' = newest first.

## `dateFrom` (type: `string`):

Only items created on or after this UTC date.

## `dateTo` (type: `string`):

Only items created on or before this UTC date.

## `minPoints` (type: `integer`):

Only stories with at least this many upvotes.

## `maxItems` (type: `integer`):

Stop after this many story or job rows (comments are counted separately, see below). Lists hold up to 500 stories; search up to 5,000 hits.

## `includeComments` (type: `boolean`):

For list and search modes: also store the comments of each story as 'comment' rows (oldest first, depth-limited).

## `commentDepth` (type: `integer`):

1 = top-level comments only, 2 = plus direct replies, … Each comment row carries its 'depth'.

## `maxCommentsPerStory` (type: `integer`):

Cost control when 'Include comments' is on.

## `hiringThread` (type: `string`):

Which monthly thread to parse.

## `hiringMonth` (type: `string`):

Pick a specific month's thread; empty = the latest one.

## Actor input object example

```json
{
  "mode": "top",
  "searchTags": "story",
  "sortBy": "relevance",
  "minPoints": 0,
  "maxItems": 100,
  "includeComments": false,
  "commentDepth": 1,
  "maxCommentsPerStory": 100,
  "hiringThread": "who-is-hiring"
}
```

# Actor output Schema

## `results` (type: `string`):

All items as JSON

## `resultsCsv` (type: `string`):

Same dataset as CSV — open in Excel/Sheets

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("arthursbuisness/hacker-news-stories-comments-jobs").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("arthursbuisness/hacker-news-stories-comments-jobs").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call arthursbuisness/hacker-news-stories-comments-jobs --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,arthursbuisness/hacker-news-stories-comments-jobs"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/RxyGa1gge4QpTc6aQ/builds/bu6aqDcMsG9axjc1W/openapi.json