# Hacker News Scraper – Stories, Search & Who Is Hiring (`anxuanng/hacker-news-scraper`) Actor

Hacker News stories, comments and Who Is Hiring job posts as JSON. Read the front page feeds, search by keyword and date, or turn the monthly hiring thread into rows with company, role, location, remote, salary and stack. Only-new mode for alerts.

- **URL**: https://apify.com/anxuanng/hacker-news-scraper.md
- **Developed by:** [Xuan An Nguyen](https://apify.com/anxuanng) (community)
- **Categories:** News, Jobs, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.00 / 1,000 row scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Hacker News Scraper – Stories, Search & Who Is Hiring

Get Hacker News as clean JSON in three ways:

- **Feeds:** the front page (top), new, best, Ask HN, Show HN and YC job lists, with rank, points and comment counts.
- **Search:** stories or comments by keyword, newest first or best match, with points and date filters.
- **Who Is Hiring:** every job post from the monthly "Ask HN: Who is hiring?" thread, parsed into company, role,
  location, remote/onsite/hybrid, employment type, salary, visa sponsorship, tech stack and apply link.

Turn on **Only new items** and schedule it for keyword alerts or a monthly jobs feed that never charges you twice.

### What data can I get?

| Field | Example |
|---|---|
| `type` | `story`, `job`, `poll`, `comment` or `hiring` |
| `title`, `url`, `domain` | `Show HN: Bodily Oddities`, the linked page, `example.com` |
| `points`, `commentsCount`, `author`, `createdAt` | `72`, `31`, HN username, ISO time |
| `feed`, `rank` | `top`, `3` (position on the list when scraped) |
| `storyId`, `storyTitle`, `parentId` | for comments and hiring posts: the thread they belong to |
| `company`, `role`, `location` | `Modash.io`, `Senior Product Engineer`, `Remote (Europe)` |
| `remote`, `onsite`, `hybrid` | `true`, `false`, `false` (all `null` when the post doesn't say) |
| `employmentType`, `salary`, `visaSponsorship` | `Full-time`, `€75k–110k`, `null` |
| `technologies`, `applyUrl` | `["Python", "Rust"]`, the careers link from the post |
| `headline`, `text` | first line and full text of the post as plain text |
| `comments` | with **Include comments**: the thread as `{id, parentId, depth, author, text, createdAt}` |
| `hnUrl` | link to the item on news.ycombinator.com |

Every row has the same columns. Fields that don't apply to a row type are `null`.

### How much does it cost?

**$1 per 1,000 rows.** Include comments adds $1 per 1,000 stories that have comments. You pay only for rows
delivered. The full Who Is Hiring thread for one month (~250–400 posts) costs about $0.25–0.40.

### How do I use it?

Front page right now:

```json
{ "mode": "feed", "feeds": ["top"], "maxItems": 30 }
```

Every comment that mentions your product, newest first, as a scheduled alert:

```json
{ "mode": "search", "search": "yourproduct.com", "searchIn": "comments", "sort": "date", "onlyNew": true }
```

Remote Rust and Elixir jobs from the last three hiring threads:

```json
{ "mode": "hiring", "hiringMonths": 3, "remoteOnly": true, "keywords": ["rust", "elixir"] }
```

### How good is the Who Is Hiring parsing?

The thread asks posters to start with `Company | Role | Location | REMOTE/ONSITE | ...`, and most do, loosely. On the
September 2026 thread (257 posts), company came out on 94% of posts, location and work mode on 89%, a link on 86%
and a role on 70%. Salary shows up on the 28% of posts that list one. Posts that ignore the format get `null` fields
rather than guesses. `headline` and `text` always carry the post exactly as written, so nothing is lost.

### Use cases

- **Job seekers and job boards:** a monthly, structured feed of startup jobs with salary and stack.
- **Recruiters and sales:** companies hiring engineers right now, with their stack and careers page.
- **Brand and launch monitoring:** new comments or stories that mention your product, company or domain.
- **Research and newsletters:** top stories with points and comment counts, or a topic's history by date.

### FAQ

**Where does the data come from?** The official Hacker News API (Firebase) for feeds and items, and Algolia's public
HN Search API for search, hiring threads and comment trees. No login and no HTML scraping. Posts belong to their
authors: if you show them elsewhere, link each one back to its `hnUrl`.

**What about personal data?** Usernames are the public HN handles shown next to each post. The "Who wants to be
hired?" and "Seeking freelancer" threads, where people post their CVs, are left out of every mode.

**Why does best-match search stop at 1,000?** Algolia returns at most 1,000 results per query in relevance order.
Sort by **Newest first** to page back through everything.

**Do I need a proxy?** No. It runs at 256 MB without a proxy.

### More scrapers by the same developer

- [ATS Jobs Scraper](https://apify.com/anxuanng/ats-jobs-scraper): Job postings straight from company career boards on Greenhouse, Lever, Ashby, Workday, Workable, Recruitee and SmartRecruiters in one schema, with salary ranges and an only-new mode.
- [Chotot Scraper](https://apify.com/anxuanng/chotot-scraper): Scrape Chợ Tốt (chotot.com), Vietnam's largest classifieds site: cars, motorbikes, real estate, phones, laptops and more, with prices in VND, location and all item attributes. Pay per listing.
- [EU Company Registry Scraper](https://apify.com/anxuanng/company-registry-scraper): Companies from the official open registries of France (SIRENE), Norway (Brønnøysund) and Finland (PRH): legal form, status, activity code, address, workforce and French revenue. New-registration monitoring.
- [Events Scraper](https://apify.com/anxuanng/events-scraper): Upcoming events in any city from Eventbrite, Meetup and Luma in one schema: time in UTC, venue, online or in person, organizer, price and attendance.
- [Prediction Markets Scraper](https://apify.com/anxuanng/prediction-markets-scraper): Kalshi and Polymarket markets in one schema: yes/no probabilities, bid/ask, volume, liquidity, close time and results, with keyword and category filters and an only-changed mode for monitoring.
- [Remote Jobs Scraper](https://apify.com/anxuanng/remote-jobs-scraper): Remote job listings from Himalayas, Remote OK, We Work Remotely, Jobicy and Arbeitnow in one schema, deduplicated across boards, with salaries and an only-new mode.
- [SEEK, JobStreet & JobsDB Jobs Scraper](https://apify.com/anxuanng/seek-jobs-scraper): Job listings from SEEK (Australia, New Zealand), JobStreet (Malaysia, Singapore, Philippines, Indonesia) and JobsDB (Hong Kong, Thailand) in one schema, with salary, work type, classification and an only-new mode.
- [Substack Scraper](https://apify.com/anxuanng/substack-scraper): Posts from any Substack newsletter with likes, comments, word count, free or paid flag and the full text of free posts. Search publications by keyword; only-new mode for monitoring.
- [Telegram Channel Scraper](https://apify.com/anxuanng/telegram-channel-scraper): Scrape public Telegram channels: posts, views, reactions, media, links, forwards, plus channel stats. Incremental mode for daily monitoring. Pay per post.
- [Vietnam Jobs Scraper](https://apify.com/anxuanng/vietnam-jobs-scraper): Scrape job postings from VietnamWorks, TopCV and ITviec in one unified schema. Extract title, company, salary in VND/USD, location, level, skills, deadline and full description. Pay per job, JSON/CSV export, API and MCP access.

# Actor input Schema

## `mode` (type: `string`):

'Feeds' reads the front page lists (top, new, best, Ask, Show, jobs). 'Search' finds stories or comments by keyword and date. 'Who Is Hiring' returns the job posts from the monthly Ask HN thread, parsed into company, role, location, remote, salary and tech stack.

## `feeds` (type: `array`):

Feeds mode: which lists to read. Rows alternate between feeds, and a story on two feeds is returned once.

## `search` (type: `string`):

Search mode: words to look for, e.g. 'postgres', 'claude code', 'your-company.com'. Leave empty with 'Newest first' to read everything posted recently.

## `searchIn` (type: `string`):

Search mode: stories (titles, links and Ask/Show text) or comments. Comments are the place to catch mentions of a product or company.

## `sort` (type: `string`):

Search mode: best match (up to 1,000 results) or newest first (no limit).

## `hiringMonths` (type: `integer`):

Who Is Hiring mode: how many monthly threads to read, newest first. 1 is the current month.

## `keywords` (type: `array`):

Keep rows whose title, text or link contains any of these words (whole words, any case), e.g. 'rust', 'python', 'berlin'. Works in every mode. Empty keeps all.

## `remoteOnly` (type: `boolean`):

Who Is Hiring mode: keep only posts that say remote in their first line.

## `sinceDays` (type: `integer`):

Skip anything older than this many days. Empty keeps all.

## `minPoints` (type: `integer`):

Stories only: skip stories with fewer points.

## `minComments` (type: `integer`):

Stories only: skip stories with fewer comments.

## `fetchComments` (type: `boolean`):

Add each story's comment thread (author, text, time, reply depth) to its row. Charged as a separate event, only for stories that have comments.

## `maxCommentsPerStory` (type: `integer`):

With 'Include comments': stop after this many comments per story, read top threads first.

## `onlyNew` (type: `boolean`):

Skip anything you already received in an earlier run of this Actor. Turn on for scheduled monitoring (keyword alerts, new job posts) so you pay only for new rows.

## `maxItems` (type: `integer`):

Stop after this many rows. You are charged only per row delivered.

## Actor input object example

```json
{
  "mode": "feed",
  "feeds": [
    "top"
  ],
  "searchIn": "stories",
  "sort": "relevance",
  "hiringMonths": 1,
  "remoteOnly": false,
  "fetchComments": false,
  "maxCommentsPerStory": 100,
  "onlyNew": false,
  "maxItems": 100
}
```

# Actor output Schema

## `listings` (type: `string`):

All scraped rows as JSON; append ?format=csv for CSV.

## `runSummary` (type: `string`):

Counts of pushed/skipped/failed rows and whether the budget limit was hit.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "mode": "feed",
    "feeds": [
        "top"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("anxuanng/hacker-news-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "mode": "feed",
    "feeds": ["top"],
}

# Run the Actor and wait for it to finish
run = client.actor("anxuanng/hacker-news-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "mode": "feed",
  "feeds": [
    "top"
  ]
}' |
apify call anxuanng/hacker-news-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,anxuanng/hacker-news-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/LEearPNvaTroC4CHA/builds/ZGcSR8VuXg9XHlWkI/openapi.json