# Hacker News Scraper & Who Is Hiring Jobs Parser (`sysco32/hacker-news-scraper`) Actor

Scrape Hacker News front page, new, best, Ask, Show and job stories, full-text search with points and date filters, comments, and the monthly 'Ask HN: Who is hiring?' thread parsed into structured jobs (company, role, location, remote, salary). Official APIs.

- **URL**: https://apify.com/sysco32/hacker-news-scraper.md
- **Developed by:** [Gabor Sziladi](https://apify.com/sysco32) (community)
- **Categories:** News, Jobs, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.30 / 1,000 item results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Hacker News Scraper & "Who Is Hiring?" Jobs Parser

Scrape **Hacker News** with the official HN API and HN Search by Algolia: the **front page** and the new, best, Ask HN, Show HN and job lists, **full-text search** with points and date filters, **comment trees**, and the monthly **"Ask HN: Who is hiring?"** thread **parsed into structured jobs** (company, roles, locations, remote, salary, visa sponsorship, technologies and links). Fast, cheap (**$0.30 per 1,000 items**) and with no browser.

- ✅ Story lists in live ranking order: top (front page), new, best, ask, show, job
- ✅ Search stories or comments by keyword, with minimum points or comments, a date range, and relevance or newest-first sorting
- ✅ Comments for every story, depth-first in thread order, with `depth` and `parent_id`
- ✅ **Who is hiring parser**: one clean record per job post, with salary ranges normalised to `{min, max, currency, period}`
- ✅ Filters: keywords, exclude words, remote only, locations, only posts with salary
- ✅ **Only new items** monitoring mode for keyword alerts and new hiring posts. You pay only for new items.

### Input examples

Front page:

```json
{
  "mode": "stories",
  "storyLists": [
    "top"
  ],
  "maxItems": 30
}
```

Remote jobs with salary from this month's "Who is hiring?" thread:

```json
{
  "mode": "whoishiring",
  "hiringThread": "latest",
  "remoteOnly": true,
  "onlyWithSalary": true,
  "keywords": [
    "python",
    "rust"
  ],
  "maxItems": 500
}
```

A keyword alert (schedule it every hour):

```json
{
  "mode": "search",
  "searchQuery": "postgres",
  "searchSort": "date",
  "minPoints": 10,
  "maxItems": 50,
  "onlyNewItems": true
}
```

### Output example (real run)

A story:

```json
{
  "record_type": "story",
  "list": "top",
  "rank": 1,
  "id": 50180873,
  "title": "PHP 8.2.35",
  "url": "https://github.com/php/php-src/releases/tag/php-8.2.35",
  "domain": "github.com",
  "author": "tuyuri",
  "points": 852,
  "num_comments": 479,
  "created_at": "2026-10-21T13:45:33Z",
  "text": null,
  "hn_url": "https://news.ycombinator.com/item?id=50180873",
  "parent_id": null,
  "story_id": null,
  "dead": null,
  "scraped_at": "2026-10-21T19:03:26Z"
}
```

A parsed "Who is hiring?" post (`record_type: hiring_post`, text shortened):

```json
{
  "record_type": "hiring_post",
  "id": 49922609,
  "thread_id": 49922569,
  "thread_title": "Ask HN: Who is hiring? (October 2026)",
  "thread_month": "October 2026",
  "author": "mcintyem",
  "posted_at": "2026-10-01T15:03:36Z",
  "company": "Sidekick Labs",
  "roles": [
    "Senior SWE, R&D Engineer, Intelligence Engineer"
  ],
  "locations": [
    "REMOTE (US Timezones)"
  ],
  "remote": true,
  "workplace_types": [
    "remote"
  ],
  "employment_types": [
    "full-time"
  ],
  "salary": {
    "min": 170000,
    "max": 230000,
    "currency": "USD",
    "period": "year",
    "text": "$170k to $230k"
  },
  "equity": true,
  "visa_sponsorship": null,
  "technologies": [
    "Python",
    "Go"
  ],
  "urls": [
    "https://www.linkedin.com/in/three-two-one",
    "https://sidekicklabs.com"
  ],
  "header": "Sidekick Labs | Senior SWE, R&D Engineer, Intelligence Engineer | REMOTE (US Timezones) | Full-time | $170k to $230k Base + Equity + Health",
  "text": "Sidekick Labs | Senior SWE, R&D Engineer, Intelligence Engineer | REMOTE (US Timezones) | Full-time | $170k to $230k Base + Equity + Health\n\nSidekick builds AI security software for enterprises. Our sensors observe AI…",
  "hn_url": "https://news.ycombinator.com/item?id=49922609",
  "replies": 0,
  "scraped_at": "2026-10-21T19:06:23Z"
}
```

Use the **Stories & comments** or **Who is hiring: jobs** dataset views, or export JSON, CSV or Excel. The `OUTPUT` record has per-task counts and errors.

### Pricing

Pay per event, no subscription:

| Event | Price |
|---|---|
| Item result (story, comment, job or poll) | **$0.0003** ($0.30 per 1,000) |
| Parsed hiring post | $0.002 ($2 per 1,000) |
| Actor start (per run) | $0.002 |

The default run (front page, 30 stories) costs about $0.011. A whole monthly "Who is hiring?" thread (about 250–400 posts) costs about $0.50–0.80. Filtered-out items are never charged, and you can set a maximum cost per run.

### Use cases

- **Tech job boards and newsletters**: structured, filterable jobs from the monthly HN hiring thread.
- **Recruiting and sales intelligence**: which startups are hiring, where, for which stack, and at what salary.
- **Brand and keyword monitoring**: get every new story or comment that mentions your product.
- **Trend research**: top stories, domains and discussions over time.
- **AI and RAG datasets**: clean HN discussions with thread structure.

### FAQ

**Which APIs are used?** The official Hacker News API (`hacker-news.firebaseio.com`) for story lists and items, and HN Search by Algolia (`hn.algolia.com/api`) for search, comment trees and hiring threads. Both are public, read-only and need no login. Requests are rate-limited politely.

**How accurate is the hiring parser?** HN posts are free text. Most follow the `Company | Role | Location | REMOTE | Salary` header convention, and the parser reads that line and then scans the body for salary, visa and technologies. Fields it can't find are left `null` or empty, and the full `text` and the `hn_url` are always included so you can check.

**Can I get older hiring threads?** Yes. Set `hiringThread` to a month such as `September 2026`, a thread URL or ID, or use `latest` with `hiringMonths: 6` to parse the last six threads.

**How does "Only new items" work?** Delivered item IDs are remembered per mode in a named key-value store (`stateStoreName`), and later runs skip them. Run the hiring mode daily and you only get, and pay for, posts added since the last run.

### Related actors

More Actors by the same author:

- [Substack Scraper (sysco32/substack-newsletter-scraper)](https://apify.com/sysco32/substack-newsletter-scraper): posts, likes, comments and full text of free posts from any public Substack newsletter.
- [Remote Jobs Scraper (sysco32/remote-jobs-feed)](https://apify.com/sysco32/remote-jobs-feed): remote jobs from Jobicy, Remote OK, Arbeitnow and company career pages.
- [ATS Jobs Scraper (sysco32/ats-jobs-feed)](https://apify.com/sysco32/ats-jobs-feed): jobs from 9 applicant tracking systems in one schema, with ATS auto-detection and only-new-job alerts.
- [RSS Feed Reader & Monitor (sysco32/rss-feed-reader)](https://apify.com/sysco32/rss-feed-reader): RSS, Atom and JSON Feed items as clean JSON, with only-new-item monitoring.

### Need a custom scraper or data pipeline?

If you need data from another community or news site, extra fields, or the results delivered to Google Sheets, a database or an n8n workflow on a schedule, I take custom projects on Fiverr: [Python scraping scripts and n8n automation workflows](https://www.fiverr.com/sysco32/build-python-scraping-scripts-and-n8n-automation-workflows). Public data only; message me with the details first and I'll confirm whether it's doable.

# Actor input Schema

## `mode` (type: `string`):

Choose one mode. Each mode uses the options in its section below.

## `maxItems` (type: `integer`):

Maximum stories per list, search results, or parsed hiring posts (comments are counted separately).

## `storyLists` (type: `array`):

Lists from the official HN API, in their current ranking order.

## `searchQuery` (type: `string`):

Full-text search via HN Search by Algolia, e.g. <code>postgres</code> or <code>"rust compiler"</code>. Leave empty to list everything that matches the filters.

## `searchTags` (type: `string`):

Which kind of HN items to search.

## `searchSort` (type: `string`):

Relevance ranks by text match and points. Newest first is best for monitoring.

## `minPoints` (type: `integer`):

For story lists and search.

## `minComments` (type: `integer`):

For search.

## `createdAfter` (type: `string`):

ISO date, e.g. <code>2026-01-01</code>.

## `createdBefore` (type: `string`):

ISO date, e.g. <code>2026-10-01</code>.

## `hiringThread` (type: `string`):

<code>latest</code>, a month such as <code>October 2026</code>, or a thread URL/ID.

## `hiringMonths` (type: `integer`):

With <code>latest</code>: also parse the previous N-1 monthly threads.

## `remoteOnly` (type: `boolean`):

Keep only posts that mention REMOTE in their header line (and don't say 'no remote').

## `locations` (type: `array`):

Keep only posts whose header line contains one of these strings, e.g. <code>London</code>, <code>Berlin</code>, <code>EU</code>, <code>NYC</code>.

## `onlyWithSalary` (type: `boolean`):

Keep only posts where a salary or pay range was found.

## `includeText` (type: `boolean`):

Add the full plain-text post. Turn off for a compact dataset.

## `keywords` (type: `array`):

Keep only items (or hiring posts) whose title or text contains at least one of these words, e.g. <code>rust</code>, <code>python</code>, <code>LLM</code>.

## `excludeKeywords` (type: `array`):

Drop items containing any of these words.

## `itemIds` (type: `array`):

e.g. <code>https://news.ycombinator.com/item?id=8863</code> or <code>8863</code>.

## `includeComments` (type: `boolean`):

Also output the comment tree of each story (lists, search and items modes), depth-first in thread order.

## `maxCommentsPerStory` (type: `integer`):

Upper limit of comments per story when comments are enabled.

## `onlyNewItems` (type: `boolean`):

Remember the item IDs already delivered in this mode and skip them in later runs. Great for keyword alerts (search sorted by date) and monthly hiring-thread updates: you pay only for new items.

## `stateStoreName` (type: `string`):

Named key-value store for seen IDs. Use a different name for each independent monitor.

## Actor input object example

```json
{
  "mode": "stories",
  "maxItems": 30,
  "storyLists": [
    "top"
  ],
  "searchTags": "story",
  "searchSort": "relevance",
  "minPoints": 0,
  "minComments": 0,
  "hiringThread": "latest",
  "hiringMonths": 1,
  "remoteOnly": false,
  "onlyWithSalary": false,
  "includeText": true,
  "includeComments": false,
  "maxCommentsPerStory": 100,
  "onlyNewItems": false,
  "stateStoreName": "hacker-news-scraper-seen"
}
```

# Actor output Schema

## `results` (type: `string`):

Stories, comments, jobs and polls.

## `hiring` (type: `string`):

Parsed 'Ask HN: Who is hiring?' posts: company, roles, locations, remote, salary, visa, technologies.

## `summary` (type: `string`):

Records per task, failed tasks with error messages.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "mode": "stories",
    "maxItems": 30
};

// Run the Actor and wait for it to finish
const run = await client.actor("sysco32/hacker-news-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "mode": "stories",
    "maxItems": 30,
}

# Run the Actor and wait for it to finish
run = client.actor("sysco32/hacker-news-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "mode": "stories",
  "maxItems": 30
}' |
apify call sysco32/hacker-news-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,sysco32/hacker-news-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/fThivT1QVyMhevN5q/builds/7MD2kLQtpb3rWVzcR/openapi.json