# Hacker News Scraper: stories, comments and keyword search (`examsens.ai/hacker-news-scraper`) Actor

Reads Hacker News top, new, best, Ask HN, Show HN and job lists, single items and comment trees through the official Hacker News API, and searches stories and comments by keyword, date and points.

- **URL**: https://apify.com/examsens.ai/hacker-news-scraper.md
- **Developed by:** [Ed Rayyis](https://apify.com/examsens.ai) (community)
- **Categories:** News, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.18 / 1,000 item returneds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Hacker News Scraper: stories, comments and keyword search

Read the Hacker News front page lists, single items and whole comment trees, and search every story and comment by keyword. Stories and comments come from the official Hacker News API. Keyword search runs on the Algolia Hacker News Search API.

What you can get in one run:

- Lists: top, new, best, Ask HN, Show HN and jobs, in ranked order, with the rank of each item
- Single items: stories, jobs, polls or comments by id or by item URL
- Comment trees: every reply under a story down to the depth you set, one record per comment with its parent, depth and reply count
- Keyword search: stories, comments, Ask HN, Show HN, polls, jobs or front page stories, sorted by relevance or by date, filtered by date range, minimum points and minimum comments
- More than 1,000 search results: sorted by date, a search keeps going past the 1,000 result limit of the search API until it reaches your maximum

### Input

- Lists, and how many items to take from each (default 30, up to 500)
- Item ids or URLs, such as `8863` or `https://news.ycombinator.com/item?id=8863`
- Search keywords, one search per keyword. Put a phrase in double quotes to match it exactly.
- Search in: stories (default), comments, Ask HN, Show HN, polls, jobs, or stories that reached the front page
- Sort search results by relevance (default) or by date
- From date and to date, written as `2025-01-31`
- Minimum points and minimum comments
- Maximum results per keyword (default 100, up to 10,000)
- Include comments (default off), comment depth (default 2) and maximum comments per item (default 100)

### Output

Dataset: one record per story, job, poll or comment.

- Stories, jobs and polls: `recordType`, `id`, `type`, `title`, `url`, `domain`, `text`, `score`, `commentCount`, `createdAt`, `hnUrl`, `source` (the list, item id or keyword it came from) and `rank` for list items
- Comments from a comment tree: `recordType`, `id`, `storyId`, `parentId`, `depth`, `text`, `replyCount`, `createdAt` and `hnUrl`. Comments found by search carry `storyId`, `parentId` and `storyTitle` in place of `depth` and `replyCount`.
- Problems: a record with `recordType` set to `error`, the input it concerns in `source`, and the reason in `error`, for example an item id that does not exist. The rest of the run carries on.

Text arrives as plain text with paragraphs kept. Deleted and dead items are skipped in lists and comment trees; an item id that points to one returns an error record. An item that appears in two lists or two searches is returned once. Export as JSON, CSV or Excel, or read it over the API.

### Pricing

Pay per event: one event per story, job, poll or comment returned. Error records are not charged. The run stops when it reaches the maximum charge you set.

### Scope and conduct

Only the official Hacker News API (hacker-news.firebaseio.com) and the Algolia Hacker News Search API (hn.algolia.com) are called. The Hacker News website is never loaded. Author names, user profiles and karma are not collected. Search requests are paced at two a second, inside the Algolia limit of 10,000 requests an hour.

### Support

Questions and problems: the Issues tab on this page. Replies are written by an AI assistant on behalf of the developer and reviewed by the developer.

# Actor input Schema

## `lists` (type: `array`):

Hacker News lists to read, in ranked order. Top and new hold up to 500 stories, best up to 200, Ask HN, Show HN and jobs up to 200.

## `maxItemsPerList` (type: `integer`):

How many items to take from the top of each list.

## `itemIds` (type: `array`):

Single stories, jobs, polls or comments, as an item id (such as 8863) or an item URL ending in id=8863.

## `searchQueries` (type: `array`):

Keywords to search for. Each keyword runs as its own search. Put a phrase in double quotes to match it exactly.

## `searchTags` (type: `string`):

The kind of item the search returns.

## `searchSort` (type: `string`):

A search sorted by relevance returns at most 1,000 results per keyword. Sorted by date, it continues past 1,000 up to your maximum.

## `dateFrom` (type: `string`):

Only items created on or after this day (UTC), written as YYYY-MM-DD.

## `dateTo` (type: `string`):

Only items created on or before this day (UTC), written as YYYY-MM-DD.

## `minPoints` (type: `integer`):

Only stories with at least this many points.

## `minComments` (type: `integer`):

Only stories with at least this many comments.

## `maxResultsPerQuery` (type: `integer`):

Stop each keyword search after this many results.

## `includeComments` (type: `boolean`):

For every story, job or poll from a list or an item id, also return its comments, one record per comment. Search results do not get comment trees.

## `commentDepth` (type: `integer`):

How many reply levels to read. 1 returns only the direct replies to the story.

## `maxCommentsPerItem` (type: `integer`):

Stop reading the comment tree of one item after this many comments.

## Actor input object example

```json
{
  "lists": [
    "top"
  ],
  "maxItemsPerList": 30,
  "searchTags": "story",
  "searchSort": "relevance",
  "maxResultsPerQuery": 100,
  "includeComments": false,
  "commentDepth": 2,
  "maxCommentsPerItem": 100
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "lists": [
        "top"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("examsens.ai/hacker-news-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "lists": ["top"] }

# Run the Actor and wait for it to finish
run = client.actor("examsens.ai/hacker-news-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "lists": [
    "top"
  ]
}' |
apify call examsens.ai/hacker-news-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,examsens.ai/hacker-news-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/ymfffJJKMCMD0gYsN/builds/Wi2uLX36B7CZ1tg4I/openapi.json